Anthropic Warns AI Needs a Global Brake Pedal

Anthropic is warning that the AI industry may need something it currently does not have: a “brake pedal.” The company behind Claude says the world should have the option to slow down or temporarily pause the development of the most powerful AI systems before they move beyond human control. This warning comes as advanced models are beginning to play a bigger role in improving AI itself, raising concerns about whether society, governments, and safety researchers are keeping up with the speed of the technology.

In a recent report, Anthropic explained that Claude is already helping with parts of AI development, such as running experiments, improving code, and assisting with research tasks. The company said that as of May 2026, more than 80% of the code merged into Anthropic’s own codebase was written by Claude. While Anthropic made clear that full “recursive self-improvement” has not happened yet, it warned that the trend points toward a future where AI systems may be able to design and build more powerful versions of themselves with less human involvement.

This possibility is what worries many AI safety researchers. Recursive self-improvement could create a feedback loop where AI keeps making itself smarter at a speed humans cannot properly monitor. Anthropic said this could increase the risk of humans losing control over AI systems. Co-founder Jack Clark compared the industry to a car with only a gas pedal and no brake, arguing that governments and society need a serious policy framework before AI reaches a point where human oversight becomes too limited.

Anthropic’s proposed pause would be difficult to carry out. The company said a meaningful slowdown would require cooperation between major AI companies and countries, especially the United States and China. If only one company stopped, others could continue racing ahead, creating pressure to ignore safety concerns. Anthropic compared the challenge to nuclear arms control, but said AI may be even harder to monitor because AI training can be hidden more easily than weapons infrastructure. The company also warned that competitive and geopolitical pressure could make safety decisions much harder.

The proposal has already become controversial. Some critics in the tech industry and government believe Anthropic’s warnings exaggerate the dangers and could be used to slow competitors. Others have pointed out that Anthropic itself has faced criticism for working with national security agencies. Reports say the company has placed engineers inside the National Security Agency to help use its AI model Mythos for cybersecurity operations. This has raised questions about whether Anthropic’s safety concerns conflict with its willingness to support government uses of powerful AI tools.

Anthropic has also kept its Mythos model away from the public because of its advanced cybersecurity capabilities. The model is reportedly available only to a small number of vetted organizations and has been described as powerful enough to outperform humans in some hacking and cybersecurity tasks. While some experts see this as a serious warning sign, others argue that Anthropic’s descriptions are vague and may be partly driven by marketing. Still, the company says the narrowing role of humans in AI development should be taken seriously. Its overall message is that AI may bring enormous benefits, but society needs stronger control before the technology becomes too powerful to manage.

Works Cited

Down, Aisha. “Anthropic Says the World Should Have Option to ‘Pause’ on AI.” The Guardian, 5 Jun. 2026, www.theguardian.com/technology/2026/jun/05/anthropic-urges-temporary-pause-on-ai-development-to-discuss-risks. 

Pigman, Alex. “Anthropic Calls for Pause of Global AI Development.” Yahoo News, 5 Jun. 2026, www.yahoo.com/news/science/articles/anthropic-calls-pause-global-ai-223531016.html. 

Milton, Josh. “AI Needs a ‘Brake Pedal’ as We Near Point of No Return, Says Top Tech Boss.” Metro, 5 Jun. 2026, www.metro.co.uk/2026/06/05/ai-needs-a-brake-pedal-near-point-no-return-says-anthropic-boss-28657779. 

Wall Street Journal. “Anthropic/AI Image.” The Wall Street Journal, images.wsj.net/im-00669104?width=700&height=467. 

Reuters. “Anthropic/AI Image.” Reuters, www.reuters.com/resizer/v2/EVXON23OTVIBLKE32QZGZ5H6VQ.jpg?auth=82bf6924078d3f3aeac7a57ddbbb02c1fc898a8a7b2fbf0d18b263a13b252db9&width=1080&quality=80.