Anthropic PBC has issued an urgent call for a coordinated global pause on advanced AI development, warning that systems are nearing a point where they could improve themselves without human oversight.
In a blog post, the company’s head of internal research Marina Favaro and head of policy Jack Clark argued that frontier models are approaching “recursive self-improvement”-the ability to write their own code and expand capabilities. They say this threshold could be reached in as little as two years, risking massive societal disruption.
“We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development,” the authors wrote, to allow societal structures and alignment research to catch up.
Anthropic, which recently raised $65 billion at a near-trillion-dollar valuation and confidentially filed for an IPO, acknowledged that enforcing a global pause would be extremely difficult. The company compared it to Cold War-era nuclear treaties but noted AI training runs are far easier to hide than missile silos. International cooperation, including with China, would be essential.
Critics, including venture capitalist David Sacks, accuse Anthropic of pushing a “regulatory capture agenda”-using fear to promote heavy-handed rules that would favor its proprietary models over cheaper open-source alternatives.
Despite the challenges, Anthropic said its Anthropic Institute will continue collaborating on verification systems and pushing for a broader global debate among policymakers and researchers.