Kubernetes has given platform teams a consistent way to deploy, scale, and operate containerized applications. Now, many of those same teams are being asked to support AI. The transition is already underway. According to the CNCF 2025 Annual Cloud Native Survey , 66% of organizations hosting generative AI models use Kubernetes for some or all of their inference workloads. Yet only 7% of organizations deploy AI models daily. That gap highlights an important distinction: running AI on Kubernetes and having a Kubernetes platform ready to operate AI continuously are not the same thing.
The challenge is also showing up within platform teams. The 2025 State of AI in Platform Engineering research found that 35% of platform teams still don't orchestrate AI workloads, pointing to a gap between AI adoption and the operational platforms needed to support it at scale. AI doesn't require platform teams to abandon cloud native practices. Kubernetes, GitOps, observability, automation, and self-service remain valuable foundations. But AI introduces new requirements around compute, scheduling, model delivery, and operations.
AI is often discussed as a GPU workload, but production AI pipelines are heterogeneous. Data preparation, preprocessing, retrieval, orchestration, and application logic may run on CPUs, while training or inference uses GPUs or other accelerators. A single workload may depend on several resource types. Platform teams need to consider accelerator type and availability alongside CPU, memory, topology, and workload characteristics. Kubernetes is evolving to support this. Dynamic Resource Allocation (DRA), for example, provides a more flexible, declarative way for workloads to request specialized hardware.
The goal isn't simply to make GPUs available. It is to make a heterogeneous compute part of a consistent Kubernetes resource model. Cloud native teams have made application delivery repeatable through CI/CD and GitOps. AI adds another critical artifact: the model. Code + model + configuration → evaluate → deploy → observe → update Models can be large, depend on specific runtimes or hardware, and require evaluation before deployment. Platform teams, therefore, need a clear way to answer the following questions: Which application, model, and configuration are running, and can we reproduce the deployment?