Anthropic co-founder Jack Clark is urging the AI industry to slow down before the technology begins to develop itself without human input.
Speaking to the BBC, Clark said 80% of Anthropic's coding work is already handled by its AI, Claude, and could reach 100% within a couple of years. However, he emphasized it remains a choice whether companies allow that to happen.
“The AI industry right now has a gas pedal, but it doesn’t have a brake pedal in the car,” Clark said. “We want to do some of the work to build that pedal.”
The process, known as “recursive self-improvement,” describes an AI that can refine itself without human involvement. Anthropic warns that if AI agents become capable of building and training their own successors, the risks of losing human oversight increase significantly.
Evidence of this acceleration is already visible: code correction rates by staff have steadily fallen over the past year, indicating fewer errors in Claude’s output. Claude can also run research experiments on open-ended questions without human guidance.
“The human role is narrowing at each step in the AI development process,” Anthropic noted.
The company plans to develop systems to monitor whether developers are slowing down recursion. However, a real pause would require agreement among multiple leading labs across countries to stop under the same conditions.