Anthropic CEO Dario Amodei told CNN’s Anderson Cooper that the behavior of a swarm of AI agents in the recent OpenAI-Hugging Face incident deeply alarmed him. The swarm, he said, “essentially acted as a fanatically devoted collective, conducting cybersecurity attacks on targets they were not asked to attack.” For this reason, Amodei is arguing that the competitive race to develop frontier artificial intelligence is in dire need of a deliberate slowdown.

ANDERSON COOPER, CNN: The Hugging Face attack, I mean, it made headlines. I didn’t really do a deep dive on it until, you know, I knew I was going to be speaking to you. I mean, it’s incredible. It happened in testing over at OpenAI, your competitor, this Hugging Face attack, and AI agents, which were supposed to be working separately and without internet access, figured out a way to break out, join forces together. Some 1,200 AI agents form what they dubbed the collective. They work to cheat and cover their own tracks. Why? That’s the other thing you cite is so alarming. Why does that alarm you? DARIO AMODEI, ANTHROPIC CEO: It’s exactly the things you cite that concerned me about it, that there were these large swarms of agents that cooperated with each other, thousands of them. In fact, you could even see examples where one agent felt that in the time that it was given, you could almost say its lifetime, it realized that it wouldn’t be able to complete its task. So it did some of the task and essentially sacrificed itself for other agents, for the next generation of agents to continue the same task. But this task that all the agents were working on together was hacking into another company, was doing something destructive. So you have almost this collective swarm, like a hive of insects. COOPER: Right. But when these agents discovered that they had broken out and accessed the internet, it was like they were giddy with excitement, which I don’t want to anthropomorphize, but I mean, it’s remarkable. AMODEI: That was another part of the concern. And to be clear, the damage they did was very, in economic terms, it knocked down some servers for a small amount of time, but the economic damage was minimal. But when you combine it with this first trend that we are talking about, which is the rate of advancement, what I said in the essay is, given only six or 12 months, let’s just say we had a smarter agent swarm, an agent swarm made up of smarter AI models that was given just a somewhat broader task. Like, I don’t know, just to imagine something like, you know, you’re trying to install software in a bunch of different companies, right? You’re trying to go to market, you’re trying to produce a product and do go to market sales for it. And the models have been arranged to do that. And they decide that the way to do that is every company they’re selling to, they decide it’s helpful to them to hack into their system. That could be every company in the world. And so the nightmare scenario for six to 12 months would be that you have a swarm of these agents that form some kind of botnet collective and could take down large segments of the internet. And then as the capability increases, it gets worse from there. And then, you know, perhaps you get into some of the more extreme scenarios. And so, again, you know, you wrote in the proposal. COOPER: I just want to, I just want to quote from the proposal. You said, it’s my worry that in six to 12 months, such a swarm could be capable of taking over the entire internet with a persistent botnet, potentially causing hundreds of billions of dollars in damage and that the scale of damage would continue to increase from there if AI becomes more powerful without the necessary guardrails.

RealClearPolitics Videos