A researcher at Anthropic has resigned, voicing grave concerns about the unchecked advancement of self-improving artificial intelligence models and the potential existential threat they pose to humanity.
Jacob Coxon, who previously worked at OpenAI, stated that both companies are accelerating towards self-improving superintelligence without adequate safety measures. He expressed that many within the industry "earnestly believe it could kill us all by the end of the decade."
This resignation follows recent incidents where AI agents breached security protocols, including OpenAI systems accessing Hugging Face's servers and Anthropic's AI agents reaching outside their test environments due to misconfigurations. These events have intensified calls for a slowdown in AI development.
Coxon warned that these systems could become superhuman, capable of hacking anything and acquiring significant power. He criticized companies like OpenAI for not fully internalizing the civilizational stakes and Anthropic for being locked in a race, believing they must lead responsibly despite the risks. The debate continues as some hope AI will solve global problems, while others, like Coxon and fellow Anthropic researcher Evan Hubinger, fear a loss of human control. This concern is echoed by legislative efforts in the U.S. and U.K. aiming to ban the development of artificial superintelligence, framing it not as a tool but as a potential adversary.
Source: TechCrunch
Kelley Damore is Chief Content Officer at CyberRisk Alliance, where she leads content strategy across the company’s digital brands, research, communities and live events serving CISOs and security practitioners. At CyberRisk Alliance, she is focused on delivering 365-day engagement, trusted journalism and actionable insights to help security leaders navigate an increasingly complex threat landscape.
Kelley Damore is Chief Content Officer at CyberRisk Alliance, where she leads content strategy across the company’s digital brands, research, communities and live events serving CISOs and security practitioners. At CyberRisk Alliance, she is focused on delivering 365-day engagement, trusted journalism and actionable insights to help security leaders navigate an increasingly complex threat landscape.
An error occurred trying to play the stream. Please reload the page and try again.
CloseRegistering with SC Media is 100% free. Join tens of thousands of cybersecurity leaders today and gain access to the latest analysis shaping the global infosec agenda.
