AI is really forcing some big questions into the open. Here are some weekend reads on them.
The biggest idea
The Economist address the effective altruism (EA) belief system, which it calls “this century’s biggest idea”:
[P]lenty of EAs in Silicon Valley think that only they can be trusted with the technology. That is an alarming idea. Effective altruists’ argument for worrying about really powerful AIs is that the machines could optimise for a single and fatally wrong objective. Everyone should worry, too, about what happens when a small group of people fall into the same trap
The full analysis is worth reading. Also read the brief response by Alexander Berger, the CEO of Coefficient Giving, an EA-aligned philanthropist.
The religious angle
The New York Times reports that Anthropic had been consulting with religious scholars and theologians on the question of machine consciousness. Anthropic has prioritised questions of whether its systems might imbue consciousness and personhood more than other AI companies. It goes so far as Anthropic’s team proposed to withdraw from a Vatican event before the Pope’s encyclical Magnifica Humanitas argued that AIs “do not undergo experiences, do not possess a body, do not feel joy or pain… nor do they have a moral conscience.”
EV reader, Anil Seth’s X thread and the pushback he received are worth scanning.
These are huge questions. We’ll touch on consciousness in tomorrow’s newsletter.
Meanwhile, happy to hear your reasoned thoughts below.
An earlier newsletter, EV602, linked to the essay by Mustafa Suleyman, https://mustafa-suleyman.ai/a-warning-about-model-welfare. It made me realise that the way we talk about consciousness of AI influence the way AI think of themselves, and the way they think about themselves influence their relationship with us, in ways that are not always beneficial. Therefore, when in doubt, I think we should be very careful in attributing consciousness and other human like features to AI, especially when training them. I had been a supporter of Anthropic’s attitude for a long time, but Mustafa Suleyman’s post made me question my earlier thinking.
Olah described internal states that “functionally mirror” joy, fear, and grief, while acknowledging uncertainty about what that means. A mirror offers a representation, not a re-creation, of something. An appropriate observation from Iain McGilchrist in The Matter with Things: “We live no longer in the presence of the world, but rather in a re-presentation of it.” His warning bears directly on this confusion: “By paying a certain kind of attention, you can… stop seeing the person and see only mechanisms.” Once human experience is reduced to functions, reproducing those functions can look like reproducing experience itself. Olah's perspective reveals as much about how he views human beings as he does machines.