Anthropic warns AI may soon begin recursive self-improvement
Anthropic warns AI may soon begin recursive self-improvement
Publish Date: 2026-06-05 14:00:00
Source Domain: www.scientificamerican.com
The companies at the frontier of artificial intelligence should be ready to slow down, one of the fastest-moving among them says.
Anthropic, the maker of the Claude chatbot, claims AI systems may be on the cusp of what it calls recursive self-improvement—the point at which they can design and build their own successors with little human input. The company says this could increase the risk of humans losing control of the technology.
“We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology,” Anthropic said in a June 4 blog post titled “When AI Builds Itself.”
On supporting science journalism
If you’re enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.
The proposal highlights a tough problem in AI governance. A slowdown would need rival companies and governments in several countries to accept the same limits at the same time, with no treaty obliging them and competition only intensifying. That makes the warning technically important and politically fraught: Anthropic is calling for the brakes on a race in which it remains a front runner.
The speed at which the technology is developing could have “huge implications” on society. Anthropic points to its own operation as a warning sign. The company says Claude now writes more than 80 percent of the code merged into its systems, up from low single digits before it released Claude Code in early 2025, and that its engineers ship around eight times as much code per quarter as they did a few years ago. At each step of building AI, it argues, the human role is shrinking. “We are not there yet, and recursive self-improvement is not inevitable,” the company said. “But it could come…
