The Bottleneck Moved

For the better part of three years, the defining question in digital infrastructure has been some version of: how much compute can we build, and how fast can we build it. Gigawatt campuses. GPU allocations measured in the tens of thousands. Power purchase agreements negotiated years in advance. The entire industry organized itself around a single scarce resource, and the companies that controlled access to it set the terms for everyone else.

That question is starting to answer itself. It's not that compute has become abundant, capacity constraints are still real and still make headlines, but the industry has spent three years throwing capital at the problem, and the picture looks different than it did in 2023. What's changing faster than the supply of compute is the shape of the demand for it.

Training built the story. Inference is rewriting it.

Training a model is, economically speaking, a one-time event. You commit a large, predictable amount of compute over a defined window, the model comes out the other end, and the spending largely stops. It behaves like a capital investment: expensive, but finite.

Inference doesn't behave that way. Every query, every chatbot exchange, every AI-generated recommendation consumes compute again, continuously, for as long as the product exists. Deloitte's most recent estimate puts inference at roughly two-thirds of all AI compute in 2026, up from about a third in 2023 and half in 2025. Multiple industry analysts now describe inference costs exceeding training costs within weeks of a product's launch, not years. The Futurum Group has gone further, projecting that inference will overtake training in revenue terms this year.

That's not a rounding error. It's a structural shift in what the infrastructure has to be good at.

A different workload wants a different building.

Training tolerates distance. A cluster training a frontier model doesn't care whether it's sited next to the people who will eventually use the model, it cares about power, cooling, and enough internal bandwidth to keep GPUs fed. That's why the last three years of site selection have optimized, correctly, for cheap power and available land, wherever that happened to be.

Inference is a live conversation, not a batch job. It has a user on one end of it, waiting on a response, and the physics of that exchange are unforgiving: every additional hundred miles between the model and the person asking it something adds latency that shows up as a worse product. Real-time inference, agentic workflows, anything voice- or video-adjacent, punishes distance in a way training simply never did. The workload that's growing fastest is also the workload most sensitive to where it physically sits.

That single fact is quietly redrawing the map. Some industry observers are already calling network infrastructure the constraint that matters now, more than GPU availability, more than power and cooling alone. The companies that will serve this next wave of demand well are not necessarily the ones with the most compute. They're the ones who've figured out how to get intelligence to where the questions are actually being asked.

The next scarce thing isn't capacity. It's proximity.

None of this means the megawatt campus era is over. Training still needs massive, concentrated, power-rich sites, and that need isn't shrinking. But it does mean the industry's next competitive edge is unlikely to be won on the same terrain as the last one. If inference is where the growth is, and inference lives or dies on latency, then the facilities that matter most in the next phase are the ones built or positioned to sit close to where people actually are, not the ones that can claim the most contiguous acreage.

That's a genuinely different design problem. It asks different questions at the site selection stage. It rewards different kinds of assets. And it suggests that the next round of value creation in this industry won't go exclusively to whoever has the most compute, it will go, in meaningful part, to whoever has figured out how to close the distance between that compute and the people using it.

The industry spent the last three years solving for scarcity. The next few years look like they'll be spent solving for distance.

Part 2 looks at why inference workloads behave fundamentally differently than training, and what that means for how these facilities need to be built.

‍