Hysteria about the present dangers of artificial intelligence is hitting new peaks after several AI lab heads sounded the alarm and an Anthropic employee resigned out of concern that developers are not instituting appropriate safeguards against the destructive capabilities of AI.
Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman both warned that the pace of development should be slowed in order to ensure that models are properly aligned with human values and objectives.
Jacob Coxon, the researcher who resigned from Anthropic claimed on Twitter/X that, “The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible – but I hear the same people express fear privately. No other human activity poses this level of danger.”
Another Anthropic employee, Evan Hubinger, added that, “Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade.”
So can AI really kill us all in 10 years and is there an immediate existential threat that needs to be remediated without delay?
Congress is facing growing pressure to act. For example, Senator Bernie Sanders, D-Vermont, is looking to introduce legislation to permanently ban superintelligent AI, which seems like a short-sighted move that would undercut America’s AI capabilities. Does Sanders suggest that we allow nations like China to surpass us in the AI race?
Such a misguided proposal seems to be prompted by the panic induced by these apocalyptic scenarios coming from developers. Luckily we have reason to believe that such predictions are absurd.
What method could current AI models use to destroy humanity within the next 10 years?
Perhaps they could gain access to our nuclear weapons and cause a worldwide nuclear armageddon, or perhaps they could bioengineer powerful pathogens that would wipe us all out.
Both cases assume that AI will have much greater capabilities than they currently have. These scenarios also require that humans act extremely irresponsibly. AI systems would have to be integrated into nuclear launch facilities in the first case; in the second case, the laboratory process of creating pathogens would have to be optimized for use by AI. A RAND report found that these apocalyptic scenarios are exceedingly unlikely.
Remember that according to some Anthropic employees, these nightmare scenarios have a higher than 10% chance of happening – how do you go about assigning such an absurdly high probability? Plausibly, it’s not because they fear current AI, but because they fear recursive self-improvement generating a superintelligence. It seems that most of the more apocalyptic concerns are coming from researchers who are specifically worried about recursive self-improvement rather than the misuse of AI by humans.
AI developers are attempting to create AI that can genuinely improve upon itself. Theoretically, such an AI would create a successive chain of improving iterations that would eventually lead to a superintelligence – such a model would have capabilities far beyond human understanding and our ability to control it would prove extremely difficult if not virtually impossible.
So if current AI models aren’t really an existential threat, perhaps AI developers are getting close to solving recursive self-improvement, which could pose a legitimate threat. So far, there is little evidence that developers are close enough to such a milestone to warrant the claim that we’re in imminent danger.
A self-improving AI would need to demonstrate that it’s capable of solving novel and open-ended problems about AI design, for which it doesn’t have the benefit of training or the ability to simply look up an answer – this is required because the creation of a superintelligence would need AI to have superhuman abilities to innovate AI development.
Current research that examines progress in AI being able to carry out novel AI development indicates that we aren’t particularly close. AI models are not close to being able to replicate human-level AI development research, which means that the chain of improvement is missing its first link. The point isn’t that self-improving AI is impossible (although it’s not a given) but that there isn’t enough evidence that we’re close enough to that milestone to support apocalyptic claims.
Even setting long-term fears of apocalypse aside, nearer-term doomerism also seems suspect.
Just earlier this month, Amodei warned that “in 6–12 months … a swarm could be capable of taking over the entire internet with a persistent botnet – potentially causing hundreds of billions of dollars in damage.”
It’s unclear why Amodei thinks this will be possible so soon given that the UK-based AI Safety Institute found Mythos, an advanced Anthropic model, to be capable of compromising only “small, weakly defended and vulnerable” systems. Niels Rogge, a scientist at Hugging Face, called Amodei’s claim, “bizarre nonsense” and indeed it looks like Amodei and other developers have few reservations about making unjustified and irresponsible claims.
As with most fantastical, over-the-top, panic-inducing claims, these should be taken with a few grains of salt. There are reasons to be skeptical of the motives of AI developers and their constant warning about how dangerous their products allegedly are. AI developers have a financial interest in keeping the AI hype-train rolling – and what better way to do so than to attribute humanity-destroying abilities to their models?
To be sure, there is enough uncertainty about AI’s future that we should be taking the possibility of cataclysmic events seriously, and legislators and AI labs will need to work together to ensure that we develop AI responsibly. We can do that without traumatizing the population with a persistent background fear that civilization may end at any moment.
Rafael Perez is a columnist for the Southern California News Group. He recently received a doctorate in philosophy from the University of Rochester. You can reach him at rafaelperezocregister@gmail.com.