Growing calls around the world to slow or halt the development of artificial intelligence (AI) are colliding with rapid advances that are bringing the technology closer to developing future versions of itself with limited human involvement.
Anthropic's Report Highlights Autonomous AI Progress
US technology firm Anthropic recently added to those concerns by publishing a report suggesting the threshold for autonomous AI development may be closer than previously believed. The company said more than 80 percent of the code powering its Claude chatbot is now generated by the model itself, compared with a share in the single digits in early 2025.
According to Anthropic's AI development framework, current technology has reached the fourth of five stages, where models can autonomously execute code and delegate tasks to other systems. The fifth and final stage is expected to involve autonomous agents capable of designing and training their own models, potentially allowing future AI systems to continuously develop successor versions.
Warnings from Experts and Public Figures
The developments come despite repeated warnings from academics, technology leaders and public figures about the risks associated with increasingly advanced AI systems. One of the most prominent warnings came in March 2023, when the Future of Life Institute published an open letter signed by figures including Elon Musk, calling for a six-month pause on training AI systems more powerful than GPT-4. The letter warned that AI systems approaching human-level intelligence could pose significant risks, including massive job losses, widespread misinformation and a potential loss of human control over civilisation.
Pope Leo XIV also addressed the issue in his encyclical Magnifica Humanitas, released in May, calling for a halt to AI development and urging countries to separate the technology from military and economic competition.
Global Investments Drive AI Forward
The scale of global investments into the sector is one of its primary obstacles in slowing the rapid development despite these global pleas for caution. Global spending on AI is projected to rise 47 percent year-on-year in 2026 to $2.59 trillion, before increasing to $3.49 trillion in 2027.
