The Prime Minister has admitted there could be ‘risks to national security’ after a safety researcher at Anthropic issued a bleak warning.
Researcher Evan Hubinger warned he believed there was a greater than 10% chance that AI could wipe out humanity within 10 years.
It was also revealed that Anthropic, which owns Claude AI, withheld its latest model from the UK’s AI Safety Institute, one of the world’s leading bodies for testing the risks of the technology.
Burnham said: ‘Hubinger is absolutely right that AI poses risks to our national security, but it also could be the source of solutions to keeping us safer.’
Hubinger shared his candid thoughts in a post on social media after resigning from Anthropic.
‘We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to,’ he said.
Another researcher, Jacob Coxon, warned that a race between companies to self-improving superintelligence was ‘gambling with our lives’, as he resigned from his role at the firm.
Announcing his departure, Coxon said that Anthropic and OpenAI, its main competitor, were not ‘acting responsibly’ and that people outside of AI labs were underestimating the power of the technology.
‘The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. No other human activity poses this level of danger,’ he added.
Former Cabinet Office minister Darren Jones called for a new multinational treaty to regulate the development of the technology after the researchers’ warnings.
In a letter to the Prime Minister, UN Secretary-General António Guterres and OECD Secretary-General Mathias Cormann, he wrote: ‘The debate ranges from the end of humanity to claims of ‘marketing hype’ pre-IPO. Either way, governments must now step in.
‘We need a new multi-national treaty for the regulated and safe development of superintelligence. Not a ban on innovation or scientific endeavour, but a safety-first approach to the rapid development of this technology.’
A Cabinet Office spokesperson said: ‘The UK is a world leader in AI security and we have the best-funded, best-resourced security institute globally.
‘The AI Security Institute continues to collaborate closely with industry partners, including Anthropic, to make models safer. Only last week it tested OpenAI’s most powerful model, GPT-6 Astra, before public release.
‘These risks do not stop at national borders and no country can tackle them alone. The UK will continue to test the most advanced models, build a rigorous scientific understanding of their capabilities and risks, and ensure policy decisions are grounded in the evidence.’
Get in touch with our news team by emailing us at webnews@metro.co.uk.
MORE: AI safety researcher quit by telling colleagues they’re ‘causing human extinction’
MORE: Children reporting naked image sextortion scams ‘more than double’
MORE: Dakota Johnson’s Marilyn Monroe film does the impossible – I was fascinated