AI workloads are exposing a quiet gap in enterprise infrastructure that nobody saw coming—or maybe nobody wanted to see. Capacity planning, the discipline of predicting compute, memory, and storage needs before they become crises, has been effectively dormant at most organizations for over ten years. The cause isn't a skills shortage. It's institutional amnesia, baked in during an era when cloud providers convinced everyone that elasticity meant you didn't have to think ahead anymore.

How Cloud Providers Buried the Problem

For roughly a decade, capacity planning became invisible by design. AWS, Azure, and Google Cloud built auto-scaling, predictive scaling, and resource optimization tools so sophisticated that infrastructure teams stopped needing to understand their own workloads intimately. Why model your traffic patterns when a managed service could do it for you? The three hyperscalers essentially took ownership of the discipline, automating away the need for human practitioners to develop or maintain those skills.

AI Workloads Are Breaking That Model

AI inference and training workloads don't behave like traditional application traffic. They spike unpredictably, consume GPU resources in ways that standard monitoring tools struggle to track, and have cost profiles that can surprise even experienced teams. A model serving API might handle 100 requests per minute normally, then suddenly need 10x throughput during business hours with no warning. Traditional auto-scaling policies tuned for web applications simply don't account for the batch-processing patterns common in AI pipelines.

The Skills Gap Is Real (Even If Nobody Wants to Admit It)

Enterprise infrastructure teams that haven't hired capacity planners since 2015 are now scrambling to understand metrics they'd never needed before. GPU utilization, token throughput, context window sizing—these concepts weren't on most SREs' radar two years ago. The discipline didn't atrophy through neglect; it was actively discouraged by colleagues who saw capacity planning as unnecessary overhead when the cloud handled everything automatically.

What Infrastructure Teams Actually Need Now

The path forward isn't hiring expensive consultants or rebuilding everything from scratch. It starts with acknowledging that you need to understand your workloads again—really understand them, not just trust the defaults. That means investing in proper observability for AI-specific resources, building internal models of your traffic patterns, and bringing back the practice of capacity reviews that most teams abandoned when Kubernetes made deployment easier.

Key Takeaways

  • Cloud providers abstracted capacity planning so well that most teams forgot how to do it themselves
  • AI workloads expose this gap because they don't fit traditional auto-scaling assumptions
  • The solution isn't more automation—it's rebuilding foundational skills your team let atrophy
  • Start with observability for GPU and AI-specific metrics before trying to predict anything

The Bottom Line

This is a builder problem, not a vendor problem. If you've been relying on managed services to handle your resource planning, you now have a choice: learn how your infrastructure actually works again, or keep getting surprised by bills that don't match your expectations. AI didn't break capacity planning—it just reminded everyone that some fundamentals never stopped mattering.