🔮 AI hype; controlling compute; digital dialectic; a micro-dose of complexity ++ #461

An insider’s guide to AI and exponential technologies

Hi, I’m Azeem Azhar. I advise governments, firms, and investors on how to make sense of our exponential future. Every Sunday, I share my view on AI and other exponential technologies in this newsletter. I am also on LinkedIn, Threads, and Substack Notes.

---

Latest posts

If you’re not a subscriber, here’s what you missed recently:

• 🔮 Yoshua Bengio: Towards AI’s humanistic future

• 😲 My take on Sam Altman’s semiconductor ambitions: A trillion here, a trillion there. Soon, it’s real money.

• 🔮 The brilliant, complicated simplicity of ChatGPT

---

🐙 Thanks to our sponsor OctoAI. OctoAI delivers 5x more affordable LLMs at higher performance.

---

Sunday chart: Microchips over megabytes

Source: Sastry et al.

Two critical bottlenecks in developing LLMs are the scaling of computation, and the scaling of data. Focusing on these inputs is one way of effectively regulating pioneering foundational models. Data, being distributed and intangible, is hard to control. Compute, on the other hand, tangible and rooted in physical constraints, is not. Compute requires specialised hardware, such as advanced chips, produced through a highly complex and resource-intensive supply chain - a process epitomised by the cutting-edge technologies developed by companies such as ASML.

A recent research paper (co-authored by EV readers Adrian Weller & Diane Coyle) argues that these physical constraints of compute make it an ideal target for AI regulation. This approach could allow governments to:

• Monitor: By tracking computing capacity and its distribution, authorities can keep a watchful eye on AI’s evolving capabilities—and the organisations developing it.

• Allocate: There’s an opportunity to encourage AI developments that promise tangible social benefits, such as technologies to detect deep fakes.

• Enforce: This involves restricting access to essential hardware to specific groups or purposes.

The US has already started to embrace this concept, as evidenced by the AI Executive Order, which mandates oversight of models that exceed a computational power of 10^26 – a threshold nearly reached by Google’s latest Gemini Ultra model. In addition, the recent US semiconductor restrictions on China are a clear example of this enforcement in action.

The implications of surveillance and intervention in computing go far beyond AI. Computing is beginning to be seen as a public infrastructure. In my latest commentary, I highlight the fundamental role of compute in improving social welfare through wider access to information:

There are no energy-poor, information-scarce economies that have good social outcomes for their people.

However, there are risks to centralising the regulation of computing even in well-governed societies. It raises the stakes for corporate capture and may unhelpfully redirect innovation, hamper economic development, or foster a surveillance culture.

---

🚀 Today’s edition is supported by OctoAI.

OctoAI is one of the fastest, most affordable platform for generative AI inference.

Early innovators building with LLM-powered apps are flocking to open-source on OctoAI away from closed-source models like GPT for faster speeds, lower costs, and greater scalability and flexibility.

Read how OctoAI delivered 5x cost savings for LatitudeAI, makers of the hit AI role-playing game AI Dungeon while achieving superior performance at scale. Then learn how you can do the same.

---

Key reads

Are we in a hype cycle for GenAI? Are we approaching the “peak of inflated expectations” for generative AI? The FT likens the surge in Nvidia’s share price to Cisco’s unsustainable dotcom valuation. The Wall Street Journal is critical in its assessment of Microsoft Copilot, questioning its purported usefulness. It’s not uncommon for new technologies to experience a period of irrational exuberance where investment and excitement outpace the development of effective management and deployment strategies. But this hype isn’t entirely harmful. It plays a crucial role in stimulating exploration in the field, paving the way for the discovery and refinement of successful business models. (For a nuanced analysis, I recommend watching this panel discussion I was part of on hype cycles and startups.)

The hype train continues. Google has announced Gemini 1.5, a new large language model with a context window of 1 million tokens. The context window is the amount of content an LLM can consider when analysing input. Google has pushed this to 10 million tokens in internal tests. For comparison, the GPT-4 Turbo’s context window is only 128,000. The recall on it looks impressive, even superhuman. It can effectively hold an entire movie “in its head”. This opens up a whole new range of in-context learning applications. One example highlighted Gemini learning to translate from English to Kalamang by simply copying a language manual into its context window. Sounds like it might be worth the hype. From the archives: From goldfish to elephant, our commentary of Anthropic’s release of Claude-2, 14 May 2023.

But the train doesn’t stop there. OpenAI announced its text-to-video AI model Sora, which creates highly detailed video scenes up to a minute long. Again, it looks impressive and has led to a lot of speculation about what is going on (see here for a discussion of the technical report). It seems that Sora learns a physics model of the world during training. This shows the importance of scale, with more computation leading to a much better performance. OpenAI suggests that further scaling is “a promising path towards building general purpose simulators of the physical world.”On the downside, early tests of Sora against Midjourney show remarkable similarities: it seems that our AIs are reverting to a bland mean.

Both Sora and Gemini 1.5 demonstrate the power of scale but they also build further hype. Both look impressive, but we haven’t seen them in action outside of the companies’ announcements. For more insight, check out our excellent members’ discussions in the Exponential Do Slack channel if you are in Slack (apply here if not). 

Restoring the middle class. In the Industrial Revolution, it was the proletariat that needed saving. Today, in the Information Age, it seems to be the bourgeoisie’s turn. David Autor, a notable economist, points out that the advent of computers has facilitated the seamless execution of routine tasks, inadvertently leading to the devaluation of many middle-class occupations. Roles such as typists, telephone operators, and bank tellers saw a decline, while the skills of decision-makers, including lawyers and doctors, were elevated. Autor argues that AI can be the saviour of the middle class. AI can intricately “weave information and rules with acquired experience,” lowering the barriers to engaging in high-stakes decision-making. This democratisation could make such roles more accessible to a wider swathe of the middle class, albeit at the cost of potentially lowering wages. This shift mirrors what GPS has done to the taxi industry: it has democratised access to becoming a “taxi” driver by lowering the level of expertise required. But while more people earn their living ferrying passengers, they earn less on average. If we extend this storyline into the near future, we meet the automating force that is Waymo. Will AI catalyse a bourgeois renaissance, or will it spark a revolution? 

See also:

• Aneesh Raman and Maria Flynn argue that as AI advances, it will de-emphasise technical skills in favour of increasing the value of soft skills, such as relationship management.

---

Market data

Making the front of a car just 10 centimetres higher makes it 22% more likely to kill a pedestrian in an accident.

VCs returned 5.6% of their assets under management to investors last year. The lowest for 15 years.

Sheffield Forgemasters has developed a manufacturing technique that it claims could dramatically reduce the time it takes to build small modular nuclear reactors from 150 days to just 2 hours. 

The demand value of open-source software is estimated to be $8.8 trillion. 

Fervo Energy, a geothermal energy company, reduced drilling time by 70% from its first drill in 2022, demonstrating a massive 35% learning rate for drilling time improvement.

---

Short morsels to appear smart at dinner parties

✈️ What’s blocking the aircraft industry from making new commercial aeroplanes? 

🧠 Exciting week in medicine research: micro-dosing LSD enhances brain signal complexity without altering consciousness. Exercise can be one of the core methods of treating depression. AND, the first endometriosis drug may finally be on its way, as early trials show promising results. Wow. 

🤦🏽 At the other end of the spectrum, a paper with inaccurate AI-generated images of rat’s anatomy passed peer-review and was published in Frontiers in Cell and Development Biology. It was later redacted.

🤖 An incredibly dexterous humanoid robot can toast your marshmallows and do your laundry.

🦓 So, Clubhouse is still kicking: a new app feature uses AI to recreate users’ voices for a new text-to-voice experience. 

☀️ Laser light on tellurite glass creates a new type of photovoltaic cell.

🌍 Showmax’s focus on African creators and stories propels it ahead of Netflix.

---

End note

I was treated to the private view of Echoes of Living Earth by Turkish artist, Refik Anadol, earlier this week. Anadol’s immersive works are built from the first open-source, ethically curated, AI model dedicated to the natural world. He’s bridged the speed and scale of modern digital computing with the complex, interconnected elegance of nature. I’d recommend a visit to the Serpentine to experience it.

Cheers,

A

P.S. 🇰🇷 내 책, Exponential이 이제 한국어로 출간되었습니다!

---

What you’re up to — community updates

• Marko Ahtisaari becomes the new Chair of the CMI Board. Congrats!

• Sougwen Chung and Kay Firth-Butterfield received the TIME100 Impact Reward.

• Joe Cohen launched Infuse Capital and is on the lookout for new companies.

• Kris Østergaard co-authored a study on Impact Leadership in the Age of Generative AI in the Nordics.

• Steve Crossan writes about strategies founders can take to fund their ‘RealTech’ startups.

• Hana Sutch is acknowledged by the Mayor of London as one of the innovators changing how we travel around the city.

Share your updates with EV readers by telling us what you’re up to here.