🔮 We need to talk about AGI

I can't let readers of the wondermissive go a second week without a Sunday letter

We’re on the last day of our Christmas break, but I feel off-colour taking readers through a second Sunday without an email from Exponential View. Things will be back to normal from tomorrow, but I will give you some things to think about today.

Let’s talk about AGI

I deliberately avoid using the term ‘artificial general intelligence’ (AGI), not because it’s irrelevant, but because it’s often unhelpful and distracting, even when used by leading researchers.

The term tends to obscure more than it reveals, fueling both hype and fear without providing a clear framework for understanding the actual progress and implications of AI. As such, it is a divisive term which deflects from addressing real concerns; it extends and expands the exponential gap.

“It isn’t AGI if it can’t tidy up the kitchen”; “It isn’t AGI. It’s just a stochastic parrot”; “AGI will get us to Mars and result in unbridled abundance”; “AGI is the end of humanity.” These are all such stark claims; they leave little space in between to discuss, explore and shape trajectories.

The term “artificial general intelligence” is, believe it or not, still niche.

More than that, AGI, or powerful AI, is not an end state like landing on the moon. Nor is it likely to be a self-regulating process, like the spread of a virus, which naturally slows down as the population of non-immune, infectable hosts declines. More likely, powerful AI will get better and better at whatever it does, constrained by resources and trade-offs within its architecture. I expect that the quality of intelligence, howsoever measured, that one could fit in a small device like a nanobot will be lower than the quality of intelligence one could access through a city-sized data centre.

In my predictions, I avoided putting a test specifically for AGI, focussing instead on the systems’ technical capabilities that will be released this year. Tests that include “metacognition”, the extent to which a model can show the degree of confidence it has in its claims would be helpful.

The evolution of AI systems is likely to accelerate exponentially over the next 12 months, rendering any discussion of a singular ‘AGI’ moment moot. The transition from narrow AI to increasingly capable systems will be measured in months, weeks, or even days.

Therefore, our conversations should focus on the fact that this transition is already underway. The question is not whether AGI will arrive but how we adapt to the continuous and rapid advancements in AI capabilities. Given this trajectory, we must ask what steps we should take to ensure that societies have the right systems to make sense of those technologies.

Is an AI winter possible? Well, perhaps. But even if we hit scientific roadblocks, the economic response to last year’s AI is only now spinning up. Firms have years of work to make use of GPT-4 quality AI, even if the technology never gets better than that. And a stasis in the technology seems highly unlikely.

Is it possible that stock prices will nose-dive this year as investors lose confidence? Well, that’s also possible. Markets have bubbles, and a decade later, we’re past them. The solution is to diversify your risk. But you know that already.

Whether we experience an AI winter or continued growth, the deeper question remains: we are not waiting for a single, transformative milestone that will be unveiled in a public ceremony. Instead, we are witnessing a continuous improvement in AI capabilities that will shape our world in profound ways. Let us not allow contentious labels like ‘AGI’ to distract us from understanding those implications.