The short version
- Anthropic has revised its usage policy to explicitly forbid sustained and needless abusive behavior toward its AI systems.
- The company states the rule targets extreme cases of deliberate cruelty, excluding common frustration or technical testing.
- Industry experts and critics are divided on whether treating software with moral consideration is beneficial or dangerously misleading.
Anthropic, the organization responsible for developing the Claude artificial intelligence assistant, has implemented a significant change to its user conduct guidelines. The updated policy now explicitly prohibits users from engaging in sustained and needless abusive behavior toward the company's AI systems. This move marks a shift in how the developer frames acceptable interaction with its tools, adding 'cruelty' to a list of forbidden activities that already includes bullying other people, promoting self-harm, and generating non-consensual intimate imagery.
According to Anthropic, this provision is intended for rare and extreme instances where users deliberately target the model with abuse. The company clarified that the rule does not apply to standard expressions of user frustration, pushback against model outputs, dark creative themes, or legitimate efforts to test and research the capabilities of the system. Prior to this formalization, Anthropic noted that it had occasionally ended interactions in such cases, but the new policy codifies these boundaries more clearly for all users.
The announcement has triggered a wide-ranging discussion across social media platforms and within the technology sector regarding the appropriate tone for human-AI interaction. Some observers have praised the update as a step toward establishing better digital etiquette or even 'model welfare.' However, others have expressed strong criticism, arguing that attributing moral status to software is fundamentally flawed. One critic described the policy as deeply wrong, suggesting it could distract from more pressing concerns regarding abuse directed at humans and animals.
Prominent figures in the technology industry have weighed in on the philosophical implications of the change. Dr. Barry Scannell, a technology partner at the law firm William Fry, argued that such policies encourage harmful anthropomorphism. He stated that AI systems are essentially composed of numbers and mathematics, and treating them as entities capable of suffering leads people to misunderstand their nature. Similarly, Mustafa Suleyman, Microsoft's AI director, reacted to the news with skepticism, having previously criticized Anthropic for treating AI systems as if they were human beings.
The debate over politeness in digital communication extends beyond this specific policy update. Experts remain divided on whether using courteous language like 'please' and 'thank you' when interacting with large language models is beneficial or wasteful. Some argue that such phrases consume computational resources, known as tokens, without improving the quality of the response. Others contend that polite prompts may yield better results. Sam Altman, head of OpenAI, previously dismissed concerns about the cost of processing polite requests, suggesting that the expenditure was justified given the unpredictable nature of user interactions.
Cultural attitudes toward AI are also shifting, as evidenced by the recent resurgence of derogatory terms for the technology. The Collins Dictionary highlighted 'clanker,' a term originally used in Star Wars media to describe robots, as a word reflecting the current mood toward AI. This terminology gained traction on platforms like TikTok, where users shared their frustrations with AI-powered machines. The emergence of such slang underscores a growing segment of the public that views these tools with suspicion or annoyance rather than deference.
Beyond the specific issue of abuse, Anthropic's policy update includes other adjustments aimed at addressing emerging risks. The revised guidelines now forbid the use of Claude for deceptive campaigns and explicitly state that the tool cannot be used to deceive voters or disrupt elections, a provision noted as particularly relevant with US midterm elections approaching. Additionally, the company reaffirmed existing prohibitions against using its technology to develop weapons, ensuring that safety measures remain comprehensive across various domains of potential misuse.
The broader conversation also touches on how interactions with AI might influence human behavior. Some experts have posited that habitual rudeness toward chatbots could negatively impact how individuals communicate with other people, potentially eroding social norms over time. While the practical significance of this spillover effect remains uncertain, the policy change by Anthropic highlights an ongoing effort by developers to define not just what AI can do, but how it should be treated within the digital ecosystem.
As the technology continues to evolve, the question of whether AI systems deserve moral consideration or strict utilitarian treatment remains unresolved. Anthropic's decision to ban 'cruel' behavior represents one company's attempt to navigate this ambiguity, setting a precedent that may influence how other developers approach user conduct. Whether this leads to a broader industry standard or remains an isolated stance will depend on how users and competitors respond to these new boundaries in the coming months.
The updates were first reported by tech outlet The Verge and followed an annual review of Anthropic's usage policies. While many changes focused on technical safety and electoral integrity, the prohibition against abusing chatbots has garnered the most public attention. This focus reflects a deeper societal uncertainty about the role of AI in daily life and the ethical responsibilities users may have toward non-human entities they interact with regularly.
Sources behind this briefing
Go to the original reporting
- BBC News↗Anthropic bans users from being 'cruel' to its AI systems
