Good morning!
A MESSAGE FROM OUR SPONSOR
This paid post was unlocked for you by Granola, the AI notepad for back-to-back meetings.
Granola is our team’s favorite note-taker. We’re excited to partner with Granola to bring four months of free access to the Business plan (10 seats) to Exponential View annual members. This perk, valued at $560, is now part of our membership.
Upgrade to claim your free access to Granola.
The limits to recursion
What are the conditions under which AI systems could undergo recursive self-improvement (RSI) — and how long might that last? Cards on the table, I’m not wildly excited by the theory of unending accelerating recursive self-improvement for the simple theoretical issue of control and alignment. But I also think it’s not likely for practical and theoretical reasons.
Philosopher Toby Ord has done the heavy lifting for me. He argues that a key gating factor to RSI is the generation time: how long it takes of an AI system to go through a single loop of improvement (where it helps design and train its successor).
Ord concludes that while it is mathematically possible for extreme RSI, intelligence rising without bound, the conditions are unbearably difficult to achieve. The key question is whether the entire research to training to development cycle can shrink to zero or not. Ord reckons unlikely, I too don’t believe it is possible.
Generation time can’t get…
