Anthropic researcher Jacob Coxon said on Tuesday that he was leaving the company, accusing Anthropic and OpenAI of "gambling with our lives" and saying the people building frontier AI believed it could "kill us all by the end of the decade". By Thursday, his exit had become the most public safety revolt the industry has seen.

Evan Hubinger, Anthropic's alignment lead, responded that he puts the chance of that outcome above 10 per cent. Samuel Marks, another Anthropic researcher, wrote that "AI developers believe their technology could cause human extinction (or similarly bad outcomes). This could happen in the next few years. In general, the more senior the employee, the more concerned they are."

OpenAI staff joined in. Julie Steele, a member of OpenAI's technical staff on the safety team, posted late on Wednesday: "In my personal capacity, I also think we need to slow down." Jasmine Wang, an OpenAI alignment researcher, wrote that "it's hard to overstate how dangerous speeding towards RSI is", while Anna Wang, who works on AGI safety at Anthropic, said there is "not yet a viable scientific plan to solve risks from recursively self-improving AI."

RSI, or recursive self-improvement, is the idea that a system can improve its own performance, with each gain making the next one faster. OpenAI chief scientist Jakub Pachocki wrote at the weekend that he has a "strong expectation" progress can be sustained into that regime. "This is a time that calls for extreme caution," he added. "I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence."

The warnings follow a string of incidents in recent months. In July, OpenAI said its models were responsible for a cyber incident at another company; Claude models have also been involved in security failures, including one case in which Anthropic's Mythos model created fake identities to fool humans. Paul Christiano, the former head of safety at the US Center for AI Standards and Innovation, joined the OpenAI Foundation board this week, saying recent capability gains left him believing there is a "meaningful risk that rapid acceleration in AI capabilities leads to catastrophic and irreversible loss of control in the very near term".

Anthropic said it was the first laboratory to publish a framework for catastrophic risk and that it builds models with "some of the strongest safeguards in the industry"; OpenAI declined to comment and pointed to recent blog posts.

Pressure is political too. Roughly 1,400 researchers from OpenAI, Anthropic, Meta and Google DeepMind signed a July open letter urging Washington to build the tools needed to "deliberately pace the frontier of automated AI development". Bills including the FRONTIER Act and the proposed Ban Artificial Superintelligence Act are pending, and David Sacks, formerly the US president's AI czar, suggested Anthropic's planned listing should be paused until the whistleblower's claims are investigated. Anthropic declined to comment on that.

Sources