🔮 Sunday edition #521: AI MRI & agency; production capital; life after death; robots in homes, US munition & cars++

and

Apr 27, 2025

Hi, it’s Azeem with our weekend update bringing you a bit of distance from the headlines so we can see what’s really going on. Let’s dive in.

Today’s edition is brought to you by Attio, the CRM for the AI era.

Sync your emails and Attio instantly builds your CRM with all contacts, companies, and interactions enriched with actionable data.

AI’s critical crossroads

AI is transitioning from relying only on human-generated data toward autonomous learning from direct experience. Training on static human data has limits – performance eventually plateaus. DeepMind’s David Silver and Richard Sutton argue we’re entering an “era of experience,” where AI learns independently by interacting with environments. DeepMind’s AlphaProof illustrates this: it earned silver at the International Mathematical Olympiad by generating millions of its own proofs, surpassing human-data-based models.

This shift means competitive advantage moves from accumulating data to engineering interactive environments that facilitate continuous learning. Businesses should prepare for value creation to pivot towards orchestrating these dynamic learning loops.

However, increased autonomy brings significant risks. Reinforcement learning, while efficient, mostly refines existing capabilities without genuinely expanding a model’s creative problem-solving capacity. Moreover, aggressive RL optimization can lead to unpredictable behaviors, including hallucinations and unreliable self-assessment (see

Nathan Lambert

’s inspection of OpenAI’s o3). The real economic advantage will increasingly lie in robust mechanisms for verifying, controlling and safely deploying powerful AI.

Anthropic’s CEO, Dario Amodei, argues in his new essay that we are in a race between growing model intelligence and our ability to interpret AI’s inner workings. Without interpretability—the “AI MRI,” as Dario calls it—the opacity of advanced AI systems can be a significant risk, particularly in high-stakes sectors like finance, medicine and national security. Anthropic is aiming for comprehensive diagnostic capabilities by 2027 and encourages others – including competitors – to do the same.

Abundance needs capital

A few weeks ago, we wrote about Ezra Klein and Derek Thompson’s book Abundance, which argues that liberal societies must overcome the scarcity mindset and rediscover their ability to build essential infrastructure. Achieving abundance requires significant capital investment, aligned closely with technological advancements.

In his chairman’s letter to investors, Larry Fink wrote that

Decades from now, we might reflect on 2025 as another pivotal moment, when the financial landscape shifted once again.

The world needs $68 trillion for infrastructure, equivalent to rebuilding the US Interstate Highway System every six weeks for 15 years.

Financing this infrastructure is challenging. Credit funds want mature cash flows; venture capital prefers asset-light startups, leaving early-stage physical assets like factories and data centers underserved. This critical “capital orchestration gap,” as

Brett.Bivens

and

William Godfrey

note, gives rise to a new archetype of the “Production Capitalist” – firms that combine venture capital and debt financing methods to help infrastructure-focused startups structure their finances effectively. Their main role is bridging early-stage innovation with larger institutional funding sources to enable practical scaling.

CoreWeave demonstrates this hybrid approach, combining a $1.1 billion equity round with a $7.5 billion debt facility to rapidly expand GPU data centers. Hadrian similarly integrated $25 million of equipment debt into its $117 million Series B to scale aerospace manufacturing. Firms adept at this combined approach will lead the upcoming infrastructure boom, exemplifying precisely the mindset and financing needed for building abundance.

Relating to reality

Death is edging toward a subscription model. Generative AI, trained on a lifetime of texts, voice notes and video, can already sculpt interactive replicas of deceased individuals. DeepMind researchers expect that “within our lifetimes it may become common practice for people to create custom AI agents to interact with loved ones after death,” dubbing them “generative ghosts”. Monetisation has arrived first. South Korea’s DeepBrain AI charges about $12,000 to stage a single 30-minute “reunion”, plus $1,200 per repeat visit. They aren’t the only ones in the space: HereAfter AI, SeanceAI and StoryFile all promise to “connect” with the dead.

Post-mortem photography, capturing images of loved ones after death, was common in Victorian Britain – but an always-on, talking replica is a step-change in emotional stickiness. French philosopher Baudrillard wrote about simulacra, representations of reality that are becoming more “real” than reality itself. This creates a new layer of hyperreality where relationships and experiences become mediated entirely through digital simulation – and in this case could detach people from processes of grief, closure and acceptance.

Relatedly, Anthropic launched an AI Welfare program to research “model welfare.” Researcher Kyle Fish suggested that Anthropic’s Claude model might already possess some form of consciousness, assigning it a startling 15% likelihood. Neuroscientist and EV member Anil Seth urges caution – humans have a tendency to confuse AI’s linguistic fluency and problem-solving skills with genuine consciousness. Anil will write a full response to Fish on this question for members of Exponential View on Wednesday, so stay tuned!

A global recalibration

What happens when the world’s two superpowers go head-to-head in an era of economic upheaval, tech acceleration and military recalibration? This was the topic of my discussion with historian Sir Niall Ferguson. The conversation is now available on YouTube and on your favorite podcast platforms.

Elsewhere

OpenAI’s CPO says that the days of a 12-month lead are over but a 3-6 month edge matters.
Tokyo researchers created an AI model that automates fragrance production from user descriptions.
US munition stocks remain critically low compared to China’s extensive reserves.
Tesla’s robotaxi service is set to launch in Austin this summer.
The INTELLECT-2 project has launched a major effort to train a 32-billion-parameter AI model through permissionless compute contributions.
An upcoming Nvidia chip faces months of delay owing to major unresolved problems, with fingers pointing everywhere but at Nvidia. Meanwhile, Huawei is preparing to mass-ship its Ascend 910C AI chip, positioned as China’s flagship domestic alternative now that US curbs hit Nvidia H20 exports.
Generalization is a tough task for robotics, but we’re making progress:
A statistical analysis of car brands that most frequently appear in music videos. One for fun!

End note

I’ll be in the Boston area at the beginning of May speaking at MIT Technology Review’s flagship EmTech AI conference. We’re organising a get-together for members of Exponential View on Tuesday, 6th May, from 7:30pm in Boston (exact location TBC). Please RSVP here before Wednesday, 30 April.

Today’s edition is brought to you by Attio

Attio is the AI native CRM built for the next era of companies.

Sync your emails and watch Attio build your CRM instantly - with every company, contact, and interaction you’ve ever had, enriched and organized. Join industry leaders like Flatfile, Replicate, Modal and more.

Start for free today.

Exponential View