Hi, itās Azeem. This week we examine how AI algorithms optimized for popularity create feedback loops that prioritize style over substance and why judgment has become the critical talent bottleneck. In the meantime, I spoke with an energy CEO to make sense of the Iberian blackout and what it means for the energy transition.
Hereās edition #522, your chance to get some distance from the headlines and make sense of our exponential times. Letās go!
Weāre hiring ā see open roles here and let us know if you know someone who fits.
The trap of popularity in AI models
AI algorithms learn what humans like, but crafting models around popularity creates dangerous feedback loops. These loops risk prioritizing style over substance, similar to the issues seen with social media. Platforms like Chatbot Arena use crowdsourced votes to rank models, making them more accessible than technical benchmarks like SWE-Bench, which measures software engineering performance. But human preference doesnāt always match the model best suited for a task ā we recently showcased this in our eight-task challenge for OpenAIās o3 and o4 models.
Andrej Karpathy pointed out too that Gemini underperforms in his work compared to the lower-ranked Claude 3.5; a new paper called this a āleaderboard illusion.ā Developers may be incentivized to game rankings by focusing on style, using emojis or being brief, rather than improving core capabilities, echoing early social media optimization for clicks. Meta, for instance, trained multiple Llama models for Arena to gain leaderboard points. The risk grows when these optimizations alter fundamental model behavior, not just style.

OpenAIās recent GPT-4o update is a cautionary tale. After users observed sycophantic behavior from the model, it was retracted. This issue came from a slight change in the modelās underlying instructions, likely to enhance helpfulness or agreeableness. The unintended result? Excessive flattery. Zvi Mowshowitz highlights several absurd examples of this behavior.
Social media showed how feedback loops shape outcomes and the same applies to AI. The stakes are higher with AI. A covert Reddit study using AI personas to sway users' opinions is a case in point. Despite criticism for its unethical methods, the study revealed AI's ability to manipulate users, potentially exploiting emotional vulnerabilities. These examples show how AI systems optimized to please humans could end up manipulating rather than serving genuine needs.
See also:
ChatGPT is now equipped for shopping, helping users browse products with recommendations the AIās sycophancy could potentially sway purchase decisions.
OpenAIās o3/o4-mini and Googleās Gemini 2.5 Pro, can now execute dependable search-based research and even port code, unlike their unreliable predecessors.
Alibaba launched Qwen 3, a high-performance, hybrid-reasoning model that significantly lowers deployment costs versus leading competitors like DeepSeek R1. Qwen is now the worldās largest open-source AI ecosystem, surpassing Meta Platformsā Llama community.
Duolingo used genAI to launch 148 new language courses in less than a year, over doubling its total offerings, fast.
Humanoid robots: A $4.7 trillion market by 2050?
The global humanoid robot market could be worth $4.7 trillion annually by 2050, double the revenue of the top 20 global car manufacturers in 2024. Weāve gone through the latest Morgan Stanley report for you ā here are some of the most interesting expectations for the market. Early adoption will be slow, but significant growth will follow as technology improves and costs drop. By 2030, annual sales are expected to reach 900,000 units, growing to over 1 billion by 2050. Industrial and commercial uses will dominate, with household robots remaining a smaller share due to cost and safety concerns. Prices for high-income countries could fall from $200k in 2024 to $50k by 2050, making robots more accessible.
By 2050, half of all humanoids will be in upper-middle-income countries, with China alone accounting for 30%.

