🤯 Jensen Huang’s extraordinary interview

A few days ago, Jensen Huang was a guest on the BG2 podcast. I’ve heard him speak many times before, but this conversation struck me as extraordinary. We get an almost unfiltered view of Nvidia, but even more so of the trajectory of AI’s development as a technology and an industry.

On my way to Dubai, I had a chance to listen to the whole thing. This is one of the most interesting conversations I’ve listened to in a while and here I’m going to share my favourite bits¹ and takeaways… Enjoy!

The flywheel’s flywheel

Jensen spoke about how the stack of technologies that comprise AI is accelerating. Nvidia is focused on the rate of change of that acceleration… at its heart, the flywheel of machine learning helps them achieve 2-3x more performance every year.

Many people used to believe that designing a better chip with more FLOPs, more bits and bytes, was the key. You’d see their keynote slides full of charts and specs. And yes, horsepower does matter. But that’s old thinking.

In the past, software was static—just an application running on Windows. The way to improve it was to make faster chips. But machine learning isn’t human programming; it’s not just about the software. It’s about the entire data pipeline. The real key is the machine learning flywheel.

The most important part is enabling data scientists and researchers to be productive in this flywheel. It starts at the very beginning. People often don’t realise that it takes AI to curate data to teach an AI, and that process is incredibly complex. With smarter AI curating the data, we now even have synthetic data generation, adding more ways to prepare data for training. Before you even get to training, you have massive amounts of data processing involved. […]

Every step along the flywheel is challenging. In the past, we focused on making things like Excel or Doom faster. But now, you need to think about how to make the entire flywheel faster. […]

In the end, the exponential rise comes from accelerating the whole system.

Accelerating the entire system requires a holistic approach that considers Amdahl’s law. Amdahl’s law states that the overall system speedup is limited by the fraction of the system that cannot be parallelised or improved. To achieve significant acceleration, you need to optimise every component of the AI pipeline, from data preparation to inference, not just focusing on individual steps like training.

Amdahl’s law suggests that if a component in a process takes 30% of the total process time and I accelerate that component by 3x, I didn’t really accelerate that process by that much.

(“That much” being only 20% improvement in the overall system speed.)

[…] you really want to create a system that accelerates every single step because only in doing the whole thing can you really materially improve that cycle time and that flywheel. The rate of learning in the end is what causes the exponential rise.

This is Nvidia’s mission. To do all this requires an integrated ecosystem.

Nvidia’s ecosystem approach

Huang describes how the supply chain is working together to increase performance on a per dollar and per watt basis at one or two orders of magnitude faster than the equivalent time of Moore’s law.

The important thing is that Moore’s law was a social contract. It was an agreement across the range of suppliers in the semiconductor industry to step up to meet Intel’s public roadmaps. Nvidia is doing something similar but at a much larger scale.