Anthropic Warns AI Could Autonomously Build Its Successor Raising Control Risks

Source Domain: www.ndtv.com

As if it couldn’t get any more dystopian, Anthropic – the AI company behind Claude – just revealed that AI could start building itself and create its successor. The company also warned that “full recursive self-improvement also might increase the risks of humans losing control over AI systems.” Simply put, recursive self-improvement means a stage where AI is able to improve itself without any human intervention and create a more powerful successor.

The company said that as it increasingly delegates AI developments to AI systems, there are signals that with enough compute it leads to a future where “an AI system (is) capable of fully autonomously designing and developing its own successor.”

In fact, as of May 2026, more than 80% of the code merged into its
codebase was authored by Claude, the company revealed. A codebase serves as the single ‘master plan’ that developers use to build, test, and maintain the software

Anthropic, an outlier among AI companies, often warns of the ills of rapid artificial intelligence development. It said technical trends suggest AI systems are going to become much more capable in the coming years. “These trends have huge implications. AI that can build itself would be a major development in the history of technology-one that could bring enormous good for the world in science, healthcare, and beyond,” it said.

However, it warned that “If systems are capable of fully building their own successors, the ways we secure them, monitor them, and shape their behavior all grow much more important.”

Sagar Vishnoi, co-founder of Future Shift Labs also highlighted that as AI increasingly writes the code that powers future AI systems, the challenge moves from building capability to governing it. “Recursive self-improvement could significantly accelerate innovation, but it also raises important questions around oversight,…

Source