Clarifying the record on open-weight models
Anthropic founder and CEO Dario Amodei responded on Monday to murmuring in the industry that his company supports efforts by the U.S. government to ban open-weight Chinese models, or possibly open-weight models generally.
"Anyone who has read my past writing should know that I don't regard such bans as a useful measure, but let me state it clearly so that there is no doubt: Anthropic has never advocated for a ban on open-weights models," he wrote.
Taking a nuanced position
His response came after Nvidia CEO Jensen Huang posted an open letter on X in which Nvidia and a long list of other AI companies including Hugging Face, Meta, Microsoft, and Mistral urged policymakers not to impose broad "premature restrictions" on open-weight AI models. While that letter did not mention China specifically, the debate has centered on allegations that Chinese AI labs steal intellectual property from American counterparts through distillation.
Amodei said he views open-weight models as falling into a different bucket than the threat China poses. "Open-weights models that don't have dangerous capabilities are a public good: they don't cost anything besides the compute needed to run them, and they provide value to businesses, developers, and researchers."
Amodei's actual concerns
His fear is that "authoritarian governments" will build models more powerful than those in the U.S. to achieve "permanent military superiority" and/or use AI to repress their own people. He said the Chinese Communist Party isn't the only authoritarian government he's concerned about, but it is the "most capable."
Amodei also said he fears AI will enable "biological attacks" not just cybersecurity ones. And, in his view, open-weight models are more dangerous in those scenarios because it's difficult to apply guardrails.
Proposed solutions
Amodei listed actions he believes would thwart China, including restricting its access to powerful chips and calling for a formal crackdown on distillation. Interestingly, he also supports growing efforts to create a global model safety testing organization, especially if the entire world, including China, agreed to submit to it.
"Note that to be effective, testing would need to be global, which means even the CCP would need to be on board. I think this may actually be possible... limited cooperation around preventing AI biological weapons may be possible because it is in China's interest too," he wrote.
