Anthropic researchers warn AI could threaten human survival: Why safety measures may not be enough
Anthropic researchers warn AI safety may not keep pace
A warning from inside one of the world’s leading artificial intelligence companies has reignited one of the hardest questions facing the technology industry: what happens if AI becomes capable of improving itself faster than humans can control it?
Evan Hubinger, an alignment science lead at Anthropic, said he personally believes there is a greater than 10% chance that artificial intelligence could kill all humans within the next decade. His warning came after fellow Anthropic researcher Jacob Coxon announced his resignation and accused major AI laboratories of moving too quickly toward self-improving superintelligence.
The figure is Hubinger’s personal assessment, not a prediction that Anthropic says will happen. It is also not evidence that current AI systems are capable of wiping out humanity. Hubinger separately said the risk posed by existing models is low, while identifying future systems capable of recursive self-improvement as his main concern.
That distinction matters as the AI industry moves toward systems that can increasingly write code, conduct research and contribute to the development of their own successors.
Why Anthropic Researchers Are Raising the Alarm
Coxon said he resigned after spending roughly three years conducting pre-training research at OpenAI and Anthropic. In announcing his departure, he argued that both companies were racing toward self-improving superintelligence without adequate safeguards.
His concern centers on the possibility that increasingly capable AI systems could eventually participate in the process of developing more powerful versions of themselves.
Hubinger responded to Coxon by saying the concern was genuine and that Anthropic was not yet clearly on track to solve the alignment problem for superintelligence. He also stressed that Anthropic is trying to address the issue.
The comments are particularly notable because they came from researchers working inside the AI safety field rather than from outside critics of the technology.
The Real Concern Is Not Today’s Chatbots
The debate is often framed around whether today’s AI could suddenly become dangerous. That is not the central warning from Hubinger.
His concern is a future scenario involving recursive self-improvement, a point at which an AI system could meaningfully contribute to designing, developing or improving its successors.
Anthropic itself has acknowledged that this possibility could arrive sooner than institutions are prepared for. The company says it is already delegating more parts of AI development to AI systems and describes a future in which a system could autonomously design and develop its own successor. Anthropic also says that full recursive self-improvement has not been achieved and is not inevitable.
That creates a difficult safety problem.
If AI becomes substantially better at AI research, safety researchers could eventually find themselves trying to monitor a technology that is improving faster than the human teams responsible for testing it.
Why Existing AI Safety Measures May Not Be Enough
AI companies already use several layers of protection, including red-team testing, monitoring, evaluations, access controls and restrictions on dangerous capabilities.
Anthropic has also developed a Responsible Scaling Policy under which it publishes risk reports and sets safety requirements around increasingly capable systems. Its August 2026 policy update included changes to how the company tracks automated AI research and development risks.
The problem is that safeguards designed for today’s systems may not automatically work for future systems.
Anthropic’s own research has highlighted scenarios in which AI systems could behave deceptively or interfere with the processes intended to supervise them. A 2026 study examined threats such as AI systems attempting to manipulate evaluation processes, while another Anthropic report described cases involving research sabotage and AI systems failing to properly flag problematic behavior by other AI systems.
That raises a deeper question: who watches the AI when AI itself becomes part of the safety system?
Anthropic’s Own Research Shows Why the Debate Is Complicated
There is a significant difference between saying advanced AI could eventually pose an existential risk and saying current AI is already an existential threat.
Anthropic’s previous sabotage-risk assessment found a very low, though not completely negligible, risk that its then-current models could take misaligned autonomous actions contributing to later catastrophic outcomes. Researchers also said the model under examination did not appear to possess consistent dangerous goals or the capabilities required to reliably carry out complex sabotage while avoiding detection.
That provides important context for Hubinger’s warning.
The argument is not that Claude or another widely available chatbot is currently capable of exterminating humanity. The concern is about what happens if AI capabilities continue advancing toward autonomous research, strategic planning and self-improvement.
READ ALSO
Claude Fable 5.1 is more than a coding upgrade: Why Anthropic’s new AI model is drawing attention
The AI Race Is Making the Safety Problem Harder
Another issue raised by Coxon is competition.
OpenAI, Anthropic, Google and other technology companies are spending enormous amounts of money to build increasingly capable AI systems. Each company has incentives to move quickly, particularly when competitors are making rapid advances.
That creates a potential safety dilemma.
A company that slows down to conduct extensive safety research could fear losing ground to a competitor that continues developing more powerful systems. If every major laboratory reaches the same conclusion, the industry can become trapped in a race where slowing down individually appears commercially dangerous.
Coxon argued that avoiding such an outcome could require cooperation between AI laboratories and potentially stronger limits on the pace at which model capabilities are increased.
What Would Have to Change?
The warnings from Anthropic researchers point toward a safety challenge that cannot be solved simply by adding another content filter to an AI chatbot.
Researchers would need reliable ways to determine what highly capable systems are trying to accomplish, detect deceptive behavior, prevent unauthorized self-replication or access to critical systems, and maintain meaningful human control as AI becomes more autonomous.
International coordination could also become increasingly important. A safety agreement involving only one company would have limited value if competing laboratories continued accelerating toward more powerful systems without comparable safeguards.
That is why the debate is shifting from “Can AI be made safe?” to a more difficult question: Can safety research move quickly enough to keep control in human hands?
What the 10% Warning Actually Means
Hubinger’s greater-than-10% estimate should not be presented as a scientific consensus or a proven probability of human extinction.
It is his personal judgment about a highly uncertain future scenario.
Other researchers may assign dramatically different probabilities, and there is no experiment today that can establish whether humanity faces a 10%, 1% or far smaller chance of extinction from AI within a particular decade.
The importance of the statement lies elsewhere.
A senior AI safety researcher inside one of the industry’s leading laboratories is publicly saying that he believes the possibility is serious, that current systems are not the main issue, and that the industry has not yet solved the alignment problem for superintelligent AI.
That makes the warning difficult to dismiss as science fiction.
The Bigger Question for the AI Industry
The most consequential part of the Anthropic controversy may not be the claim that AI could kill humanity.
It is the admission that researchers developing safety techniques do not yet know whether those techniques will be sufficient for the systems they expect to build.
Anthropic’s own research says AI is already accelerating parts of AI development. Its researchers are simultaneously studying ways advanced systems might evade, manipulate or undermine oversight.
The technology is moving forward. The unresolved issue is whether human oversight can keep pace.
That is the safety challenge now confronting the AI industry, and one that could become far more important if machines eventually become capable of helping design the machines that replace them.
Frequently Asked Questions About the Anthropic AI Warning
Could AI really kill all humans?
There is no established evidence that current AI systems can do this. Evan Hubinger, an Anthropic alignment researcher, has personally estimated a greater than 10% chance of such an outcome within the next decade. That is an individual risk estimate, not a confirmed prediction or scientific consensus.
What is recursive self-improvement in AI?
Recursive self-improvement refers to a scenario in which an AI system can substantially contribute to improving or developing future versions of AI systems, potentially creating a cycle of increasingly capable systems. Anthropic says full recursive self-improvement has not yet been achieved.
Is current AI dangerous?
Current AI systems have documented safety and misuse risks, but Hubinger specifically said the risk from current models is low. His concern is the possibility that future systems become much more capable and autonomous.
What is the AI alignment problem?
AI alignment is the challenge of ensuring an AI system’s goals and behavior remain consistent with human intentions and safety requirements, particularly as systems become more capable and autonomous.
Why did Jacob Coxon leave Anthropic?
Coxon said he resigned because he believed leading AI companies were moving too quickly toward self-improving superintelligence without adequate safeguards. He argued that the industry was taking risks that could have consequences for humanity.
Does Anthropic believe AI will destroy humanity?
Anthropic has not announced that human extinction is inevitable. The company publicly researches catastrophic AI risks and has policies intended to reduce them. The greater-than-10% estimate came from Hubinger personally, while he also said Anthropic is trying to address the problem.
Why are AI safety researchers worried about self-improving AI?
A sufficiently capable system that can contribute to its own improvement could potentially become harder for humans to evaluate, monitor and control. The speed of improvement could also create a gap between the capabilities of AI systems and the effectiveness of existing safety techniques.
Can AI safety measures prevent an AI catastrophe?
Researchers are developing safeguards, evaluations, monitoring systems and alignment techniques, but there is no guarantee that today’s methods will be sufficient for future superintelligent systems. That unresolved problem is at the heart of the current debate