As Kubernetes fleets grow, so does the gap between the resources teams reserve and the resources their workloads actually use. Clusters running at low utilization consume compute that is paid for but never used. At the scale of hundreds or thousands of clusters, the waste compounds into millions of dollars annually. Static right-sizing workloads is one obvious remedy, but it is not the most efficient solution. Kubernetes does not ship an out-of-the-box solution that continuously tunes CPU and memory to match real demand across a large, multi-tenant fleet.

Teams that want this capability must build their own multi-dimensional scaling strategy. Salesforce was one of the first enterprise companies to pioneer Vertical Pod Autoscaler (VPA) adoption at a large production scale across the fleet along with Horizontal Pod Autoscaler (HPA). The multi-dimensional scaling strategy with a combination of both VPA and HPA optimized workload resource reservations and drove more than $5 million in cumulative infrastructure cost savings. Salesforce operates a large multi-tenant Kubernetes platform spanning over 1,400 live Amazon Elastic Kubernetes Service (Amazon EKS) clusters.

Individual clusters can be as large as 2,500 nodes and support about 1,000 workloads (Deployments, DaemonSets, StatefulSets, and more). Salesforce is one of AWS’s largest customers and we work closely with AWS to scale our Kubernetes platforms. In the platform’s early days, when the fleet was around 100 clusters and the total number of nodes and workloads was small, we managed workload right-sizing through static configurations. As our fleet rapidly grew, that operational model became inefficient and unsustainable.

We identified three key challenges with static resource management: To address these challenges and move away from static workload configurations, the Salesforce Kubernetes platform engineering team adopted the VPA to complement the HPA already running across our environments. By combining VPA for dynamic workload right-sizing with HPA for replica scaling, the team built multi-dimensional autoscaling strategies. The team has also migrated from the traditional Cluster Autoscaler to Karpenter to optimize compute resources and empower internal developers to self-serve infrastructure needs.