Self-healing GPU nodes in Kubernetes: What we learned building the EKS node monitoring agent
When you run Kubernetes at the scale we do on Amazon EKS, nodes break constantly. GPUs fall off the PCIe bus. Container runtimes wedge. Network interfaces disappear. Across tens of…