Running large-scale AI workloads on TPU and Google Kubernetes Engine (GKE) requires infrastructure that adapts as fast as your models change. Dynamic Slicing is generally available on GKE, which decouples logical TPU slice formation from physical nodepool provisioning. Compared to creating a static nodepool for each TPU slice, dynamic slicing reduces AI workload startup times by up to 5x and workload recovery by 4. 5x — maximizing TPU goodput, efficiency, and workload resilience at scale.
Previously, managing TPU capacity at GKE standard required separate, fixed TPU slice topology nodepools, increasing startup times and complexity while preventing multi-host nodepools from hosting multiple AI workloads. Changing workload needs meant manually deleting and recreating nodepools. Additionally, this tight coupling created large failure domains where a single hardware fault crashed an entire slice, slowing recovery while reprovisioning.
To overcome these challenges, we built GKE dynamic slicing, which decouples physical hardware provisioning from logical slice formation, dynamically stitching together TPU cubes using TPU optical circuit switching (OCS) technology to form various larger slice topologies (super-slicing), or partitioning TPU cubes into smaller slices for multiple workloads to run inside a nodepool (sub-slicing).
By enabling workloads to assemble custom TPU slice topologies on demand without having to delete and recreate static nodepools, dynamic slicing reduces job startup latency, limits the blast radius of hardware failures, accelerates recovery of workloads, and improves fleet utilization and efficiency.
