Anthropic moved on Thursday to bar users from treating its Claude AI system with needless cruelty, updating its usage policy to include 'a prohibition on sustained and needless abusive or cruel behavior toward our models.'
'The ability to end these interactions will remain the primary enforcement mechanism,' the policy states, per AFP — a reference to a capability Anthropic began giving Claude last year in rare, extreme cases of persistently harmful or abusive user interactions.
The update stops short of explicitly invoking 'model welfare,' the idea that AI systems might deserve some of the protections usually reserved for living things. But Anthropic and its executives have openly entertained the concept: in February, CEO Dario Amodei told The New York Times he was unsure whether models could be conscious. 'We don't know if the models are conscious... But we're open to the idea that it could be,' he said.
Not everyone agrees. In an essay last month, Microsoft AI chief Mustafa Suleyman wrote flatly that 'AIs are not conscious. They do not feel, experience, or suffer,' warning that 'granting rights and moral protections to a technological entity... is a recipe for disaster.' Pope Leo XIV weighed in from St. Peter's Basilica on Thursday, suggesting in a sermon that machines lack a soul and merely 'compile data.'
The update also adds new restrictions covering areas such as propaganda campaigns and surveillance, according to reports. Industry practitioners remain split on whether consciousness is even the right question: Jackson Stakeman of the AI services firm Sparq told AFP that 'consciousness is a trap' and that the better metaphor is a mirror — 'These systems reflect what we put in, at scale.' Whatever the philosophical verdict, the policy now makes mistreatment of the model a formal violation, an unusual governance step with little precedent among frontier labs.
