AI could wipe out humanity by launching a devastating pandemic or by causing critical water shortages, technology experts have warned, after two employees at a top AI firm warned human extinction is a possibility within ten years.
Evan Hubinger, the Alignment Science lead at Anthropic AI, has said the technology firms “really do earnestly believe AI could kill all humans”, giving it a more than one in ten chance.
“I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to,” he wrote in a post on X.
Israel sanctions row – has Burnham made his first mistake?
Andy Burnham’s Government announced a ban on trade with Israeli settlements in the occupied West Bank yesterday.
It was presented as an attempt to protect the prospect of a two-state solution, but it has already provoked threats of retaliation from Israel and drawn criticism from US politicians.
Our writers give their views on whether this is a principled attempt to restrain Israel, or an ineffective gesture.
Patrick Cockburn: Suffering makes sanctions a moral necessity
“The terrible fate of the Palestinians of Gaza and the West Bank is the great moral issue of our age.
“It should be impossible for any decent individual or government to know about the 75,000 Palestinians killed in Gaza and say ‘there was right on both sides’.
“On the contrary, this is an issue about right and wrong and the British Government is wholly correct, if late in the day.”
Jonathan Sacerdoti: This cynical move won’t help Palestinians
“The measures will do nothing tangible for Palestinians, nothing to advance peace and nothing to improve Israel’s security.
“For Britain, meanwhile, they risk damaging relations with Israel and the United States, while broadcasting a broader message of weakness: sufficient domestic political pressure can dictate British foreign policy.”
3 min read
3 min read
Paul Wood: Netanyahu’s Government has gotten exactly what they want
“If the Burnham Government hopes to change Israel’s behaviour, there may be the opposite effect to the one intended.
“The far-right members of Netanyahu’s government have spent the current election campaign looking for a fight with a foreign government – they believe they can only benefit from the ensuing controversy. Britain has now given them what they want.”
Kitty Donaldson: Greens are real target
“Burnham is seeking to distinguish himself from Starmer’s handling of Gaza and attempting to win back voters who have drifted to the Green Party. However, whether these sanctions go far enough for Green voters is another question.”
Oil prices spike after US strikes on Iranian tankers
The price of oil has risen to over $100 a barrel for the first time since July after the latest exchange of strikes between the US and Iran.
Renewed attacks
The US military said it had “destroyed” several Iranian tankers on Tuesday after Tehran attempted to strike a US navy warship with ballistic missiles.
- The Islamic Revolutionary Guard Corps (IRGC) said it struck a US military base in Jordan in retaliation. Air defence systems destroyed 18 of the ballistic missiles, with two more landing in unpopulated areas, causing no casualties, according to Jordan’s state news agency.
Rising energy prices
Brent crude, the international benchmark for oil prices, rose 2.1 per cent past the milestone amid the renewed fighting. It is up more than 60 per cent this year.
UK gas prices have also soared, with the benchmark British gas contract rising to its highest level since December 2022.
3 min read
4 min read
What this means for you
The immediate impact is the rate you can get a new fix at will get even more expensive. Then longer term, this feeds into the January price cap. It’s currently on target to rise 10% to 15% over October’s cap, and that remember is already confirmed to rise 3.6%, and was on top of the 12.6% rise in July.
money saving expert founder Martin Lewis on impact of rising gas prices
Trump’s war in Iran is driving the worst wave of pirate attacks in decades
Reduced security, new shipping routes, and desperation are fuelling a rise in attacks.
South Park TV series renamed in apparent Trump reference
The creators of the animated TV show have announced that its new series will be called South America, appearing to reference Donald Trump’s recent retitling spree.
The context
The President signed an executive order in August to rename Lake Ontario as Lake America amid trade tensions with Canada, and Apple and Google subsequently changed the name in their respective maps apps for US-based users. Trump also declared that the Gulf of Mexico should be called the Gulf of America last year.
A day before the South Park announcement, Trump shared an AI-generated video of him placing the word “America” over “Mexico” on a “Welcome to New Mexico” sign.
south park’s creators, matt stone and trey parker
History with Trump
Previously, an episode from the 27th series of the long-running show, which is known for its profanity and dark humour, depicted Trump naked in bed with Satan.
In response, the White House described as a “fourth-rate” show that was “hanging on by a thread with uninspired ideas in a desperate attempt for attention.”
Will the new name stick?
- The announcement came after won an Emmy for Outstanding Animated Program on Sunday.
- The name change shared to the show’s official social media account featured an image of the South Park sign with the word “Park” replaced with “America”.
- It remains unclear whether the new name will be adopted permanently or if it will revert to the original, with the 29th series premiering on 16 September.
All the things Trump has tried to rename – and why it smacks of desperation
First there was the Gulf of America, then Lake America.
Now, Trump has suggested renaming the state of New Mexico as ‘New America.’
UK politics
State pension triple lock could be axed – how to replace it
Andy Burnham is facing growing pressure to scrap the triple lock on state pensions to help repair the nations’ finances.
Here’s how the Government could navigate any journey away from the policy – and what could replace it.
Why might the triple lock be axed?
The British Chamber of Commerce (BCC) has, for the first time, called on Chancellor John Healey to abandon the policy, under which the state pension rises by whichever is highest of inflation, wage growth or 2.5 per cent each year.
The BCC argued that replacing it with an inflation-matched raise (a single lock) could save £3.3bn in two years. Currently, the triple lock costs £146bn a year.
Other options to replace triple lock
Smoothed earnings link
Mostly following wage growth but rising with inflation if there’s a temporary price shock.
Double lock
Removing the 2.5 per cent guarantee, rising with the higher of inflation or wages.
People build up credits through work and activities, and retirement age is determined by individual health.
Why is it controversial?
The Labour Party committed to keeping the triple lock in its 2024 manifesto. Going back on this would likely alienate older voters, who are generally more likely to go to the polls in an election than younger generations.
2 min read
3 min read
‘The triple lock has got to go’
There is no longer a financial or moral justification for keeping the triple lock, but scrapping it represents a political risk that no prime minister has felt like taking.
It was only introduced to rebuild the value of the state pension after decades in which it had fallen relative to earnings – not necessarily as a permanent settlement.
The inescapable fact is that most of us are living longer and collecting pensions for longer. Someone has to pay for that, and the bill goes to people of working age.
3 min read
Hundreds of thousands of unemployed young people may see their benefits replaced with employment support under Government welfare reforms.
Here’s what we know.
Alan Milburn, the Government’s youth unemployment tsar, suggested many young people who are not in education, employment or training (Neets) – including those diagnosed with conditions like ADHD and anxiety – should see a shift in the state support they receive. “Work beats welfare every single time,” he said.
3 min read
3 min read
Why The Netherlands is a blueprint
- Milburn was visiting The Netherlands as part of a “fact-finding mission” as he prepares to publish his final recommendations in a landmark report about young people and work.
- This year, the UK topped for the first time since 2013.
- According to research from the independent think-tank The Resolution Foundation, if Britain’s Neet rate mirrored the Dutch rate, 600,000 more young adults would be in work.
- In The Netherlands, , whereas in Britain, support is generally for unemployment and centralised via the universal credit benefits system.
High anxiety levels
Figures from the Organisation for Economic Co-operation and Development (OECD) rank the UK at the top of the charts for anxiety levels among younger people, but The Netherlands comes in second.
Milburn highlighted how this doesn’t lead to low participation levels, adding: “Just because you’ve got a condition, or even because you’ve got the diagnosis, shouldn’t mean that you aren’t able to work.”
A growing problem
Nearly a quarter – around 250,000 – of young people who are classed as Neets are estimated to have mental health conditions such as depression and anxiety, according to Department for Work and Pensions data.
The number of young people claiming Personal Independence Payments (PIP) for mental health conditions such as anxiety and neurodevelopmental conditions such as ADHD has also increased.
Don’t make these money moves before next month’s Budget
With next month’s Budget fast approaching, speculation over what the Chancellor might announce is rife.
But history suggests making moves before the red box is opened can leave people worse off. Here is what to avoid.
Taking tax-free cash early
- One of the most obvious reactions to is to take the 25 per cent tax-free lump sum before the rules potentially change.
- The Institute for Fiscal Studies proposed cutting the lump sum to £100,000 in 2024 but the change has still not been implemented.
- Ian Cook, chartered financial planner at Quilter Cheviot, warned that taking the lump sum could undermine long-term retirement plans, and means the money loses the opportunity to remain invested.
Selling assets just in case
Another rumoured Budget policy change is that the capital gains tax (CGT) rate could increase, which can tempt investors to sell assets now and pay today’s tax rate.
- However, this can mean paying more tax than needed if assets are sold in bulk, as there are multiple tax-free allowances before CGT is owed.
Giving away too much
With speculation that inheritance tax (IHT) could be raised, people often use gifting to avoid it, as gifts given more than seven years before death are exempt from IHT.
However, experts warn that rushing into this can lead to people giving away more than they can afford.
3 min read
3 min read
Making a hasty property move
There have been suggestions that the recently introduced mansion tax could be charged on homes worth £1.5m or more, rather than the current £2m.
Eamonn Prendergast, chartered financial planner at Palantir Financial Planning, said people should not panic and instead “ignore speculation when making a property purchase or sale decision, and act as if there was no speculation” rather than making a move they don’t really want.
Doing nothing at all
But experts warn that the biggest trap is becoming so worried about the Budget that you stop making sensible financial decisions altogether.
Jason Hollands from wealth management firm Evelyn Partners says: “For investors and savers, the sensible approach is to focus on decisions that make sense under current rules rather than trying to second-guess the contents of a future Budget.
Frequently consuming drinks at very high temperatures increases the risk of oesophageal cancer – cancer of the food pipe or gullet – UK research suggests.
A three times higher chance
The study, from Oxford Population Health at the University of Oxford, included nearly a million people and found those who drank “very hot” drinks had about a three times higher risk of oesophageal .
Hot liquid may damage the cell lining of the food pipe as it travels down to the stomach, which could increase the risk of cancer developing, though the overall risk is low.
3 min read
3 min read
How hot is too hot?
The research, published in the , did not record the temperature of drinks. But the International Agency for Research on Cancer has classified drinking beverages above 65°C as “probably carcinogenic”.
Drinking six or more hot beverages a day also increases the risk, data from 977,282 middle-aged men and women in this latest study shows.
What to do
- Letting drinks cool down before drinking or adding cold milk can negate the risk, experts agreed.
- Researchers said the temperature at which coffee and tea is drunk in Western populations is estimated to be .
- “More research is needed to identify the exact drink temperatures linked to increased risk,” the researchers added.
5 min read
A rare cancer
The researchers said that one or two in every 10 oesophageal cancers could be avoided if people stopped consuming hot drinks.
The cancer is rare, though, with only around 1 per cent of the UK population at risk of being diagnosed with this type of cancer.
- Other factors, such as smoking, pose a comparatively bigger risk for developing oesophageal cancer.
How to be more grateful for daily life in five easy steps
The world feels like a difficult place at the moment, and it can feel very hard sometimes to find things to be thankful for; when we are overwhelmed, we have less capacity to notice and reflect on the good things. Yet, ironically, proactively cultivating gratitude is one of the things that can actually help us feel less stressed.
NHS GP dR RADHA
The science
Research has shown that feeling or expressing thanks can actually alter our brain.
- The balance of neurotransmitters change; more serotonin and dopamine (feel good chemicals) are produced.
- This can be long lasting if we make gratitude a regular habit.
- We also get a decrease in our cortisol (stress hormone) levels which helps lower anxiety.
- Being grateful can strengthen the nerve pathways which triggers positive emotion.
How can we build gratitude?
Change your words
Instead of seeing something as a problem, try to see the opportunities within it.
Our brains are very receptive and what we read or write can alter our emotional system.
Express it
This will not only make you feel good, but you will also get a positive response from others.
How can we build gratitude?
Notice more
Throughout your day, try to notice all the things that are going well; has someone opened a door for you, or smiled?
The basics
By being thankful for the small things every day, we are far more likely to be and stay healthy so that we can enjoy them.
What to do
Gratitude can lead to feelings of optimism, lower our blood pressure, and improve sleep quality.
- Write down what potential good aspects there are to the things you don’t feel like doing, or even what could be worse about it.
- Spend a few minutes writing down three things that you are looking forward to that day, or the good things that have happened to you that day.
- Say thank you to yourself for the things you have achieved, or managed to get through and changed that day.
The warning came after Anthropic researcher Jacob Coxon quit – saying that both the company and OpenAI – the maker of Chat GPT where Coxon used to work – are “gambling with our lives”.
In a damning series of social media posts, Coxon said that “the people building AI earnestly believe that it could kill us all by the end of the decade”.
“Many executives and senior researchers will couch their phrasing in the press to sound sensible – but I hear the same people express fear privately. No other human activity poses this level of danger,” he said.
Former Cabinet Office minister Darren Jones, who left government in late July, has now said “governments must now step in” with a new multi-national treaty set-up to oversee the “regulated and safe development of superintelligence”.
But could AI really end humanity, and if so, how?
Water scarcity: The extinction war already being fought
AI experts who spoke to The i Paper mostly agreed that there is at least a risk that AI could wipe out humanity. However, they varied on the size of the risk and how any extinction could potentially happen.
Dr Alison Khan, a researcher and AI expert at Loughborough University, said the “biggest and most worrying thing that’s coming around the corner” is AI’s increasing use of scarce natural resources – particularly water – to power data centres.
AI is predicted to lead to an increase in global water usage from 1.1bn to 6.6bn cubic metres by 2027, according to analysis from the UK Government.
Sandra Wachter, professor of technology and regulation at the Oxford Internet Institute, said she believes that AI “has existential risks for humanity”.
“Data, water and electricity hungry data centres and AI systems are draining our natural resources, contributing massively to climate change and increasing costs of living for people,” she added.
“These problems are real and urgent and need addressing now.”
The four capabilities AI would need to wipe out humanity
Henry Ajder, the founder of AI-advisory firm Latent Space and an expert advisor for Meta, the UK Government and the EU Commission, said there was a “whole swathe of different ways” extinction could happen.
He said AI, which is “increasingly plugged into our systems”, could decide it is in humans’ best interests to no longer exist – and access or create chemical or nuclear weapons to achieve that.
Michael Vermeer, a senior physical scientist at research organisation RAND, has actually run a study into whether AI could exploit either nuclear war, biological pathogens and climate change to wipe out humanity.
Researchers concluded that AI would not be able to use nuclear weapons to kill all humans, but said pandemics were a “plausible extinction threat”.
However, Vermeer added: “Previous natural plagues have been catastrophic, but human societies have survived and soldiered on.
“Even a minimal population (likely a few thousand members) could eventually reconstitute the species.”
On climate change, the study found AI would have to “evade international monitoring and orchestrate” the production of “a few hundred megatons” of extremely powerful greenhouse gases to “cook the Earth to the point that there is no environmental niche left for humanity”.
The study identified four capabilities AI would need to cause human extinction. These were: an objective to cause extinction, to gain control over key physical systems like nuclear weapon launch control, the ability to persuade humans to support it, and survive without humans.
“It turns out it is very hard – though not completely out of the realm of possibility – for AI to kill us all,” Vermeer wrote in a May blog.
AI firms’ ‘arms race’ makes threat more likely
But Coxon accused the AI firms of acting irresponsibly in their approach to developing the technology – saying they are “locked in a race to get there first” rather than safely.
Anthropic, which along with OpenAI is regarded as the world’s most advanced developer of the technology, declined to offer its latest model to the UK’s AI safety watchdog for testing ahead of its release, the has reported.
The firm is preparing for a stockmarket listing in the coming weeks expected to establish its value at more than $1tn.
Despite the institute’s status as a worldwide leader in the testing of AI models, Mythos 5.1 was reportedly only offered to selected US institutions prior to release, prompting fears in Whitehall of a protectionist shift among the leading US platforms.
A Cabinet Office spokesperson said: “The AI Security Institute continues to collaborate closely with industry partners, including Anthropic, to make models safer.” Anthropic has been approached for comment.
Ajder said there was a “race to the top” amid immense competition and “hundreds of billions, if not trillions, of dollars on the line”.
“As a result, you’re seeing companies clearly pursuing research which they themselves recognise is risky if not contained appropriately,” he said.
“You also see them actively saying they are not sure if they will be able to continue to contain these technologies. I think the general consensus – that I agree with from a lot of these ex-employees – is that things are moving too quickly.”
Experts want AI companies to slow down and introduce co-ordinated safeguards to mitigate risk. But they are not optimistic.
“So few unelected people in charge of such a great power is always going to be a problem,” Khan added.
Change AI training and introduce new laws: How AI risks could be limited
Former Cabinet Office minister Darren Jones, who left government in late July, has today called for a new multi-national treaty to oversee superintelligence.
He said this would not be a “ban on scientific endeavour”, but a “safety first approach” to the rapid development of AI.
Dame Chi Onwurah, a Labour MP who chairs Parliament’s Science, Innovation and Technology Committee, told The i Paper that the Government must show it is “alive and alert” to rising AI threats.
“The government needs to show that it is ready to legislate to take on the AI tech bros and to work with other countries to do so,” Onwurah said.
She pointed out that while Donald Trump’s administration is vocally opposed to AI regulation, many US states are enacting their own laws around it.
Onwurah’s committee is pushing for an audit of the UK’s strengths and weaknesses on sovereign AI, but the MP also wants to see the UK work with the European Union and US states to develop a safety framework.
Tom Darling, executive director of The New Contract, a campaign organisation advocating for what it calls a “fair AI transition” in the UK, said: “We need to think about these AI companies like we would any other company that was being incredibly reckless.
“If you think that a jet airplanes manufacturer’s head of safety was saying ‘I’m really concerned about the direction of travel’, at the same time as like releasing a new model of plane [as has happened with Anthropic this year] everyone would say: okay, this is ridiculous. These people need to be regulated.”
Earlier this week, OpenAI’s own chief scientist Jakub Pachocki called for “extreme caution” over AI’s runaway progress and warned more intervention may be needed to ensure “humans remain in control of the future”.
“I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence,” he wrote in a blog post.
Darling suggested that AI firms should be obligated to undergo third party testing of models before they are released, or subject to a licensing regime.
There are also steps the AI firms could take to help avoid the technology spiralling out of control. Neil Lawrence, professor of machine learning at the University of Cambridge, said that AI firms should follow a “model-minimisation principle. Use the smallest model you can to do the task you need”.
Professor Maria Liakata, of natural language processing at Queen Mary University of London, added that firms should focus on producing more transparent AI models.
“There should be public investment in independent evaluation because companies cannot be open about their models as this marks a conflict of interest for them.”
‘We need a contingency plan’
On the question of whether AI will wipe out humanity, Vermeer added: “It is not absurd to say that it . At the same time, our work also showed that humans don’t need AI’s help to destroy ourselves.
“One surefire way to lessen extinction risk, whether or not it stems from AI, is to increase our chances of survival by reducing the number of nuclear weapons, restricting globe-heating chemicals, and improving pandemic surveillance.
“It also makes sense to invest in AI safety research, whether or not you buy the argument that AI is a potential extinction risk.”
Khan added: “The idea of the whole of human society being wiped out by a superintelligence has been something that has been said for at least 200 years, so it’s not a new thing. H. G. Wells was talking about this at the end of the 19th century in scientific terms.
“All the big players in AI have said it poses a massive risk. [But] we haven’t got contingency plans for this because the race has so far been more important than safety.”