Why Jacob Coxon left Anthropic over AI risk
September 10, 2026
Former OpenAI and Anthropic researcher Jacob Coxon left Anthropic and publicly warned about self-improving AI. Here is what is documented — and what his warning does not prove.
What happened
Jacob Coxon, a former researcher at OpenAI and Anthropic, has left Anthropic. In reports published on September 9, 2026, he linked his departure to concerns that leading AI labs are building increasingly capable and potentially self-improving systems while society and oversight remain unprepared. Independent outlets including AP, WIRED and TechCrunch reported on his departure and warnings.
Why the departure stands out
Coxon is not speaking only as an outside critic. He worked on powerful models and reportedly walked away before some company equity vested. That makes his decision relevant, but it does not replace technical evidence. An insider can identify an important risk and still be wrong about its probability or timing.
The core warning
The central issue is self-improving AI: a system that accelerates research, software development or experiments enough to create a feedback loop. Better systems then help build the next generation. Coxon worries that competition between labs may weaken safety brakes and that humans could lose control of exceptionally capable systems.
What we actually know
Current AI can already write code, use tools and complete multi-step tasks. It also continues to fail on apparently simple details, hallucinate and require human supervision. Coxon's departure therefore proves neither that superintelligence is imminent nor that catastrophe is inevitable. It is a serious signal about one insider's assessment of the risk.
What matters in practice
The useful response is neither panic nor dismissal. Labs can submit capabilities to independent evaluation before release, define thresholds for dangerous capabilities, report incidents and state clearly when training or deployment must stop. Governments can create transparency and liability without treating every AI application as equally risky.
Perspective
The debate becomes emotional quickly because it involves existential risk. Good decisions require reproducible tests, published safety criteria and institutions able to challenge the labs. Coxon's move increases pressure to build exactly those structures.
💡 In plain English
Imagine an engineer leaving a car company because a new vehicle is moving too quickly through development without enough brake testing. That does not prove a crash will happen, but it is a reason to take independent brake tests seriously.
Key Takeaways
- →Coxon's departure and warning have been widely and independently reported
- →His decision is a relevant insider signal, not proof that superintelligence is imminent
- →Independent evaluations, capability thresholds and stop rules are concrete responses
FAQ
Did Jacob Coxon prove that AI will cause human extinction?
No. He describes a risk and his personal assessment. It does not establish proof or a reliable timeline.
What is self-improving AI?
It means AI that greatly accelerates research and development of the next generation of AI, potentially creating a feedback loop.