🔮 Ten things I’m thinking about AI | Part 2

New scaling walls and the choice between revenue and progress

Hi all,

Ahead of ChatGPT’s 3rd birthday on Sunday, I’ve put together a four-part series on my current beliefs about AI, in which I’ll cover:

Part I: The Firm [read here, no paywall]

Part II: Physical limitations [today’s email]

Part III: The economic engine [coming up]

Part IV: The macro view [coming up]

---

Part 2: Physical limitations

That simple “exchange” with Gemini, if done by all 4.5 billion people who use Google, would use 1-1.5 gigawatt‑hours of electricity, roughly 20 minutes of London’s power demand.

And that is just the baseline. Some in our team occasionally consume 50 million tokens a day. If 4.5 billion people reached that level of use, it would exceed global annual electricity consumption.¹

This highlights the massive compute and energy load we are potentially dealing with in AI. Is there enough raw capacity to meet that demand?

Microsoft, Amazon and Google have all highlighted that extreme compute demand is causing bottlenecks. But the compute is physical: it needs to be housed in datacenters, filled with racks, cooled and powered by electricity.

At some point, you run up against physical bottlenecks. Some of this is the demand for the chips, processing and memory, evidenced by Nvidia’s reported $500 billion order backlog and the fact that SK Hynix has officially booked out its entire high-bandwidth memory (HBM) capacity through 2026. While silicon availability is one bottleneck, it is not the ultimate one.

3. The real scaling wall is energy

Energy is the most significant physical constraint on the AI build-out in the US, as I argued in the New York Times back in December. The lead time for new power generation and grid upgrades, often measured in decades, far exceeds the 18-24 months needed to build a data center. The US interconnection queue has a median wait of four to five years for renewable and storage projects to connect to the grid. Some markets report average waiting times as long as 9.2 years.

This is also a problem in Europe. Grid connections face a backlog of seven to ten years in data center hotspots.

For the Chinese, the calculus is different.

Rui Ma

points out that “current forecasts through 2030 suggest that China will only need AI-related power equal to 1-5% of the power it added over the past five years, while for the US that figure is 50-70%.”

Because the American grid can’t keep up, data center builders are increasingly opting out and looking for behind-the-meter solutions, such as gas turbines or off-grid solar. Solar is particularly attractive – some Virginia projects can move from land-use approval to commercial operation in only 18 to 24 months. Compute will increasingly be dictated by the availability of stranded energy and the resilience of local grids rather than by proximity to the end user.

These grid limitations cast doubt on the industry’s most ambitious timelines. Last year, some forecasts anticipated 10 GW clusters by 2027. This now appears improbable.