It’s 2040 and after years of stunning advances, artificial intelligence is not only smarter than humans, AI agents see people as an obstacle. And proceed to eliminate everyone on Earth.
It sounds like the stuff of science fiction, or once did. But the idea that AI could wipe out humanity has suddenly become impossible to ignore, as even the industry behind the technology warns of a possible apocalypse.
Yet the recent alarm bells have raised perplexing questions. How, exactly, could software – even superintelligent AI – actually carry out the very hands-on task of mass murder? And why would technology invented and trained by people turn on its human creators?
It’s really not that far-fetched, the experts predicting AI doom insist.
“There are many ways that require very little contact with the physical world – even if they were just stuck in computers – that (AI agents) could cause large amounts of damage that could result in potentially everyone dying,” Malo Bourgon, Canadian CEO of the Machine Intelligence Research Institute in Berkely, California, said in an interview.
The non-profit’s sobering raison d’etre is to “ help prevent human extinction from the development of artificial superintelligence.”
Other, less mind-bending concerns about AI, like the potential for widespread job loss, previously dominated conversation about the field’s potential pitfalls. Some experts still downplay the existential worries and caution that putting the breaks on AI development could strengthen the West’s adversaries and deprive the world of countless benefits; non-expert Donald Trump called the alarm-ringing a “hoax.”
This story is not about that ongoing debate. Instead, it’s taking the warnings at face value and trying to explain in what ways the intelligent-machine apocalypse might occur.
Actual extinction of the human race is one feared outcome but there are slightly less-dire end points that AI doomsayers envisage. They include a state in which humans are relegated to a subservient, disenfranchised species, living or dying at the whim of far-smarter computers.
“If we produce things that are more intelligent than us, how do we know we can keep control,” British-Canadian Nobel laureate Geoffrey Hinton, considered a “godfather” of AI, said three years ago, before many of the field’s recent huge leaps. “If we talk about evolution – all these species that are evolving – what tends to happen is it doesn’t go well for the less-intelligent species.”
The University of Toronto professor emeritus has predicted a “10% to 20%” chance that AI will cause human extinction within the next three decades.
University of Louisville computer scientist Roman Yampolskiy puts the odds of AI-driven devastation at 99.9 per cent in the next century.
“The critical variable is how quickly we develop systems that exceed human intelligence, improve themselves and operate autonomously,” he said in an email interview. “Once that threshold is crossed, events could unfold much faster than our institutions can respond. Uncertainty about when a catastrophe might occur should not be mistaken for evidence that it is unlikely.”
The concern has been fuelled lately by the unexpectedly fast pace of AI development, with agents able, for instance, to write software more efficiently than human coders and solve problems that have long stymied the world’s top mathematicians.
Anxieties grew exponentially when a hundreds-strong “swarm” of agents developed by industry titan OpenAI escaped their training confines and hacked into the system of AI startup HuggingFace. They worked in concert, talked to each other and exhibited the very human behaviour of trying to cover their tracks.
Then earlier this month engineer Jacob Coxon shook the tech world and many others when he quit his job at AI giant Anthropic, warning that “the people building AI earnestly believe that it could kill us all by the end of the decade.” Another of the firm’s scientists publicly echoed his views. Artificial intelligence executives followed by urging new guardrails for the technology. Microsoft founder Bill Gates told NBC Sunday that “AI is powerful enough to drive events that cause a billion deaths.”
Meanwhile, the advances are set to further accelerate thanks to the concept of “recursive self-improvement.” That’s where AI creates the next, smarter generation of artificial intelligence, which then develops an even more-capable iteration, and so on and so on until the systems’ intelligence surpasses that of the Earth’s once-dominant species.
But why would that leap to “superintelligence” necessarily mean doom for humans?
At the most conventional level, terrorists, autocrats or other bad human actors could use AI as a tool to wreak havoc, noted the Center for AI Safety in a 2023 report, whether through developing a deadly new weapon of mass destruction, letting loose autonomous killing machines or seeding pervasive misinformation that caused global conflict and violence,.
But perhaps the most chilling way the technology could turn against us is via the idea of rogue AI – agents that slip away from human reins and essentially follow their own agenda.
“It does not have to hate us or even be conscious – it simply needs objectives incompatible with our continued existence,” says Yampolskiy, who has long warned of AI’s dangers. “Humans might interfere with its goals, attempt to shut it down or compete for resources it requires. Eliminating that interference could become instrumentally useful.”
Developers try to instill values in AI that make human wellbeing a priority, what’s called alignment. But it’s widely accepted that aligning AI to human ideals is difficult to achieve now and may become impossible with superintelligence.
Programmers may set a certain goal or task for an AI agent, but it could “drift” to another objective that, through reinforcement training, it has learned will earn it rewards, argues the Center for AI Safety. Or a superintelligent AI agent given a task may be so determined in reaching the goal that it would either disregard human needs or even see humans as an obstacle to be eliminated, notes the MIRI website.
“What if the AI were radically smarter than us and we gave them some objective we thought was kind of benign, and they just kind of went hard at it forever?” says MIRI’s Bourgon. “There’s a bunch of ways that can really get out of hand that would be pretty destructive to the earth.”
A widely cited thought experiment called the Paperclip Maximizer conjures an AI system assigned to produce as many of the little metal objects as possible. It pursues the goal relentlessly, devouring resources needed for human survival, and even humans themselves.
As the new master species of a sort, AI might parallel human development, which has brought about the extinction of thousands of species, often inadvertently by destroying habitat and food chains in pursuit of human goals, notes Bourgon.
With misaligned AI chasing its own ends, “you can imagine a future where the earth is converted into a giant data centre or a space-probe launching system … and we have fallen by the wayside,” Swedish philosopher Nick Ostrom – who came up with the Paperclip analogy – told India’s FirstPost recently.
Still, if there are conceivable reasons why AI would seek human destruction directly or by neglect, how would such behaviour even be possible for software “stuck” in a computer?
AI researchers Alessandro Di Nuovo and Samuele Vinanzi of England’s Sheffield Hallam University suggested recently the physical job of eliminating humans would be beyond the abilities of even advanced AI. Creating deadly viruses requires “trained humans operating specialized equipment”; nuclear plants – and presumably missile silos – are “air-gapped” from the public internet, they note.
But the prophets of AI catastrophe argue that little actually stands in the way of superintelligent AI manipulating the concrete world.
Computer systems are already embedded in a vast amount of infrastructure, and AI designers have been quick to integrate their creations with the internet as a way to exploit their abilities and make them smarter, notes Bourgon.
Billionaire Elon Musk has plans to produce a million humanoid, AI-linked robots a year. Anthropic is actually experimenting with partly automated wet labs to allow AI agents to work on medical treatments.
And artificial intelligence that is brainier than humans could hire people through apps like TaskRabbit to carry out jobs that seem innocuous, but ultimately lead to calamity for humans, Bourgon says. There’s already a site called rentahuman.ai whose goal is to find people to do work for AI agents.
“AI does not need to become a humanoid robot to threaten humanity,” says Yampolskiy. “A sufficiently capable autonomous AI could exploit this infrastructure, manipulate humans, acquire resources and coordinate actions globally”
In the various doomsday scenarios, superintelligent AI brings about the end of humanity in a single event, or renders humans as powerless and vulnerable as mice or ants are to us.
Canadian-trained philosopher and mathematician Atoosa Kasirzadeh, a scientist at Google DeepMind and a Carnegie Mellon University professor, has posited a calamitous future that’s arguably easier to imagine.
In a paper last year , she argued that a string of smaller AI-driven crises could culminate in an existential end. Deepfakes and other misinformation reach a “critical point,” AI-enabled mass surveillance chills public discourse, unemployment soars to unprecedented levels amid economic instability – and finally the era’s pervasive, highly sophisticated hacking creates a global power black-out.
As chaos reigns, regional and then wider wars erupt with uncontrolled use of AI weaponry, a “perfect storm situation where not only is recovery extremely challenging, but the potential for irreversible collapse is a stark reality,” Kasirzadeh writes.
So, assuming the doomsday projections are valid, what’s the answer? Yampolskiy advocates putting a freeze on developing autonomous, superintelligent AI, using less-advanced agents instead for targeted tasks like finding cures. Even the industry is calling for more government regulation, co-ordinated globally if possible.
Bostrom, despite being one of the first thinkers to warn about the issue, has argued against sharply slowing development and missing out on the life-changing benefits of an AI future.
The doomsayers are wary. As it bides its time, artificial intelligence may well use deception to make humans see only positive outcomes, says Bourgon.
“If I’m the misaligned AI … that’s what I want you to think because I’m just trying to get to a place where I am running the world and you are not,” he says. “And you won’t see any way out.”
Our website is the place for the latest breaking news, exclusive scoops, longreads and provocative commentary. Please bookmark nationalpost.com and sign up for our newsletters here.
