🔮 Unbounded self-improvement and its limits #599

Plus: The deceptive popularity of open-weight models & DIY compute.

Aug 30, 2026

Good morning!

A MESSAGE FROM OUR SPONSOR

This paid post was unlocked for you by Granola, the AI notepad for back-to-back meetings.

Granola is our team’s favorite note-taker. We’re excited to partner with Granola to bring four months of free access to the Business plan (10 seats) to Exponential View annual members. This perk, valued at $560, is now part of our membership.

Upgrade to claim your free access to Granola.

Claim the perk

The limits to recursion

What are the conditions under which AI systems could undergo recursive self-improvement (RSI) — and how long might that last? Cards on the table, I’m not wildly excited by the theory of unending accelerating recursive self-improvement for the simple theoretical issue of control and alignment. But I also think it’s not likely for practical and theoretical reasons.

Philosopher Toby Ord has done the heavy lifting for me. He argues that a key gating factor to RSI is the generation time: how long it takes of an AI system to go through a single loop of improvement (where it helps design and train its successor).

Ord concludes that while it is mathematically possible for extreme RSI, intelligence rising without bound, the conditions are unbearably difficult to achieve. The key question is whether the entire research to training to development cycle can shrink to zero or not. Ord reckons unlikely, I too don’t believe it is possible.

Generation time can’t get to zero because real-life intrudes: experiments take time; training runs take time; making new chips take time… lots of things take time. It might still feel fast, but it wouldn’t race to infinity.

Eventually, physics intrudes too: the speed of light limits communication speed; the Bekenstein bound limits the information contained within a finite bit of space; and Landauer imposes an energy tax on irreversible computation.1

The Universe, it seems, agrees with me. Unbounded RSI has its limits.

You can read Ord’s paper here. Premium members can explore a plain English interactive version too.

The market is the instrument

Open-weight models are growing in popularity in the business world: their token share at Vercel hit a single-day record of 62%, up from 28% two months earlier. Some Western firms are even moving workloads to Chinese open weights — Thomson Reuters has developed its first in-house model based on Qwen to cut costs. You can tune cost-effective open weights to match, or sometimes beat, frontier performance on the tasks that matter to you. Take Bridgewater: working with Thinking Machines, it fine-tuned an open Qwen model on expert-labeled data and beat every frontier model it tested on its internal information-filtering tasks: roughly 30% fewer errors than the best closed model, at one-fourteenth of the inference cost. Trainloop, which I am an investor in, does something similar, using the tiny Qwen 3.7-27b model, and can outperform GPT 5.6 Sol on specific fine-tuned tasks at a fraction of the cost.

The results that Trainloop is getting are pretty impressive, seeing as they are based on a pocket model—a 27b model will even fit on a desktop Mac.2

Openweight models are, of course, getting better and better. Z.ai GLM 5.3, released this week, completely reshapes the cost-performance Pareto frontier. Of course, it isn’t a small model, but smaller distillations will emerge from it.

All of this speaks to a welcome competition in AI provision. Clearly, firms could move focused workloads onto the most-performant, fine-tuned small models they can. Where possible, they might choose large, generally capable open models. But the appeal of being at the frontier, which is more than just model performance—it is service guarantees, harness quality, reliability, and a host of other requirements—still drives significant business for Anthropic and OpenAI.

We don’t think it has much impact on the question of whether revenues flowing into the industry will materially change. For one thing, we don’t have a counterfactual to test against. But more importantly, every open model still involves paying inference providers. We’ll be looking at this question in more detail in the comings weeks.

A spicy entry to the compute ecosystem

Recursive self-improvement may be entering the compute realm. OpenAI’s new chip, ‘Jalapeño’, was designed with a heavy helping hand from the company’s own models, which helped write kernels and cut roughly 10% from one of the chip’s main compute blocks. In around 16 months from first hire to tape-out, OpenAI has built a chip that beats comparable Nvidia silicon by 1.5–1.9x on tokens per megawatt at peak throughput. This suggests frontier models can compress the design cycle for competitive silicon.

AI will result in far greater heterogeneity in chip architectures than we saw in prior computing markets. Personal computers battled between the x86 standard and the Motorola 68x, before today’s duopoly of Intel and Apple silicon. Different uses for AI will need compute optimised for intelligence, latency, power consumption, training, and inference. This creates lots of room for specialist firms. One example is that ChatGPT’s fast response mode is powered by Cerebras’ low-latency silicon. Another is Fractile, where I am an investor, which has been reported as having a deal with Anthropic for its low-latency inferencing chips.

Compute is becoming a highly segmented market, where chips aren’t a standardized commodity. This differentiated hardware demand will expand the market even if it potentially reduces Nvidia’s relative dominance.