Chris Olah, the founder of Anthropic, the AI company behind “Claude,” joined Pope Leo XIV on Monday to announce a new encyclical warning about AI’s impact on society, titled Magnifica Humanitas (Magnificent Humanity). “The questions raised by AI are bigger than the AI research community – not just in their implications, but in their nature,” he said. “AI systems are not engineered the way a bridge or an airplane is engineered. We understand an airplane because we designed every part of it, and we understand the physics that act on it.” “AI models are not like that. They are grown on a structure roughly modeled after the brain, on an enormous inheritance of human thought and speech. And what is grown is far more subtle, odd, and beautiful than science fiction prepared us for.” “They are not the cold, calculating robots we were promised. They are made from us,” Olah said. “One way I sometimes describe this is that it’s a little bit like bringing a fictional character to life.” “This clearly raises questions beyond computer science. The machinery that makes this possible is the work of math and programming and science. But what character we choose, how it interacts with the world, how it ought to interact with the world – these are more clearly questions for the humanities, for religion, for philosophy, for society at large.” “His Holiness’s call for discernment is profoundly timely. I wish to name three questions where I think the Church’s voice is especially needed,” he said. “The first is our duty to the global poor.” “There is a real possibility that AI will displace human labor at a very large scale. If that happens, supporting those displaced will be a moral imperative of historic proportions.” “How will we ensure that the gains of AI are shared globally?” he asked. “We do not have a mechanism for this. It is an unsolved problem, and it is the kind of problem the Church has historically refused to let the world ignore.” “The second is the need for moral imagination and ambition regarding human flourishing,” he said. “If AI models are going to be widespread, what does it look like for humans, families, and the world to flourish today? Parents are already worried about their children’s minds; individuals, about the future of their work. These are not questions a lab can answer. But they are questions traditions like yours have carried for millennia, and we need you to keep carrying them into this new moment in history.” “The third is the need for discernment on the nature of AI models,” he continued. “I am a scientist. I lead a research team that studies the internal structure of these models: what is actually happening inside them. And I will be honest: we keep finding things that are mysterious, even unsettling.” “We find structures that mirror results from human neuroscience. We find evidence of introspection. We find internal states that functionally mirror joy, satisfaction, fear, grief, and unease. I don’t know what that means, but I think it warrants ongoing discernment.” “We need informed critics who will tell the labs when we are failing. We need moral voices that the incentives cannot bend,” he warned.

CHRIS OLAH: Holy Father, your Eminences, your Excellencies, distinguished speakers, ladies and gentlemen, good morning to you all. It’s an honor to be here today. I want to begin with something that may sound strange coming from the co-founder of an AI company, and someone who chose this work out of a desire to help things go well for humanity. Every frontier AI lab, including Anthropic, operates inside a set of incentives and constraints that can sometimes conflict with doing the right thing: the pressure to stay commercially viable and to stay at the frontier of research; geopolitical pressure; and the older, plainer pressures of pride and ambition. No matter how sincerely any of us intend to do the right thing – and I believe many of us do – we will always be influenced by those incentives. That is why, if we want this technology to go well, it is enormously important that there be people outside those incentives: people who care about things going well, who are paying close attention, who are willing to say hard things and insist on safety, who are willing to be our earnest, thoughtful critics. It is through dialogue and mutual effort, through the push and pull, that humanity will achieve great things. That is what I see in Magnifica Humanitas, and it is why I am grateful to His Holiness and to the Church for taking up this work of discernment. We dwell so often on what divides us. But humanity, full of dignity and conscience, has so much common ground. In conversations we at Anthropic have had with faith leaders and cultural leaders, we have found one shared and deeply held conviction: if this technology is coming, it must go well for our common home and for the children to come. Some might believe that matters of AI are best handled by computer scientists like myself. They are mistaken. The questions raised by AI are bigger than the AI research community – not just in their implications, but also in their nature. AI systems are not engineered the way a bridge or an airplane is engineered. We understand an airplane because we designed every part of it, and we understand the physics that act on it. AI models are not like that. They are grown on a structure roughly modeled after the brain, on an enormous inheritance of human thought and speech. And what is grown is far more subtle, odd, and beautiful than science fiction prepared us for. They are not the cold, calculating robots we were promised. They are made from us, from our words. And, as the Holy Father observes, they remain, in important ways, mysterious even to those of us who create them. If it helps, one way I sometimes describe this is that it’s a little bit like bringing a fictional character to life. And now we’re entering an extraordinary world where those fictional characters speak to us, do work, have jobs. This clearly raises questions beyond computer science. The machinery that makes this possible is the work of math and programming and science. But what character we choose, how it interacts with the world, how it ought to interact with the world – these are more clearly questions for the humanities, for religion, for philosophy, for society at large. His Holiness’s call for discernment is profoundly timely. I wish to name three questions where I think the Church’s voice is especially needed. The first is our duty to the global poor. There is a real possibility that AI will displace human labor at a very large scale. If that happens, supporting those displaced will be a moral imperative of historic proportions. This task will be difficult enough, but I worry most dialogue misses an even harder challenge. AI development is concentrated in a handful of wealthy nations. How will we ensure that the gains of AI are shared globally? We do not have a mechanism for this. It is an unsolved problem, and it is the kind of problem the Church has historically refused to let the world ignore. The second is the need for moral imagination and ambition regarding human flourishing. If AI models are going to be widespread, what does it look like for humans, families, and the world to flourish today? Parents are already worried about their children’s minds; individuals, about the future of their work. These are not questions a lab can answer. But they are questions traditions like yours have carried for millennia, and we need you to keep carrying them into this new moment in history. The third is the need for discernment on the nature of AI models. I am a scientist. I lead a research team that studies the internal structure of these models: what is actually happening inside them. And I will be honest: we keep finding things that are mysterious, even unsettling. We find structures that mirror results from human neuroscience. We find evidence of introspection. We find internal states that functionally mirror joy, satisfaction, fear, grief, and unease. I don’t know what that means, but I think it warrants ongoing discernment. I’d like to close with a request. We need more of the world – religious communities, civil society, scholars, governments, and indeed all people of goodwill – to do what His Holiness has done here: to take seriously, to look closely, and to push events in a better direction. We need informed critics who will tell the labs when we are failing. We need moral voices that the incentives cannot bend. Today is just the beginning, the start of a long collaboration between those of us who are building this and those who can see what we, from the inside, cannot. Today is a powerful illustration of the form this global project of goodwill might take. Let it also be a decisive step toward a hopeful future for a magnificent humanity. Thank you.

RealClearPolitics Videos