AI researcher Jacob Coxon resigned from Anthropic last week, warning the industry is gambling with our lives, writing of both OpenAI and Anthropic: “Neither company is acting responsibly. The people building A.I. earnestly believe that it could kill us all by the end of the decade.” He told ABC’s “This Week with George Stephanopoulos” that he did not expect any sort of wider response, which is “really indicative of the moment being ripe for this sort of chat,” and that he felt encouraged by Anthropic founder apparently agrees with his concerns. He says he’s not an expert in foreign policy, or how the U.S. could come to a deal with China to slow down AI research, but hopes that “the people in China who are looking at the same problems we are, will come to the same conclusion that it has to be done in a measured manner.” Third-party safety monitors could “slow the ability to produce these frontier models, probably.” he said. “It’ll also ensure that there’s substantially more transparency into exactly what goes wrong if there is some sort of incident. And I think it will also allow for trust between the labs to a much greater extent if people can rely on verification from these trusted third parties.” He offered this response to the idea that Anthropic’s support for regulation is an attempt at regulatory capture: “I think actually taking this sort of measure is actively harmful for the leading frontier labs. They’re slowing themselves down more than they’re slowing their competitors down. So if anything, this is burning an advantage in order to stay safe.” “There actually wasn’t a single last straw, it really was a kind of continuous process of realization that I was genuinely not happy with the way things were going, and I needed to make like a significant change,” he said. “There were a few contributing incidents over months, but there was no like single catalyst that led to me quitting.” “Building a super intelligence is — well, it’s been likened to summoning an alien species. So I think thinking about it in terms of an alien mind that we don’t fully understand is — is pretty terrifying, even without any concrete scenario,” he said. “It’s definitely physically possible to implement a ban on artificial superintelligence if there was cooperation between all the countries. Whether or not an actual ban is a good idea is like up for discussion. I certainly agree with the proposals for slowing down seem like a very sensible first step.” “No matter what you’re currently doing, A.I. is going to have probably a significant impact on your life,” Coxon said. “And it’s pretty important to keep your eyes open about the direction things are heading and not necessarily leave it all up to the people that are building it because, you know, they want engagement. The people building the technology want to have conversations with people. And in fact, it’s been — one of the main sticking points over the last few years has been the inability to wake people up about the danger that they’re creating.”
RealClearPolitics Videos