Two prominent computer scientists from Princeton University, co-authors of the book AI Snake Oil, have expressed their views on whether artificial intelligence will kill us all by the end of the decade.
See also: itsourcecode Sales and Inventory System: Four public SQL injection vulnerabilities
The pair have become known for their work separating reality from fantasy in the world of artificial intelligence and have responded to an assertion by Anthropic that there is a greater than 10% chance that artificial intelligence will kill all humans within the next ten years.
Things started when artificial intelligence researcher Jacob Coxon, who has worked for both OpenAI and Anthropic, resigned from his job, stating that both companies knew that AI systems could kill us all by the end of the decade. His post has garnered more than 156 million views.
The people building AI honestly believe it could kill us all by the end of the decade. This is not a publicity stunt. Many executives and senior researchers may express their views more rationally in the press, but privately they express fear about the risks posed by AI.
One would expect the two companies to dismiss him as crazy, but one of Anthropic's AI security chiefs, Evan Hubinger, acknowledged that Coxon was right, albeit on a slightly longer time frame.
Hubinger stated, “Jacob is right here—we really do believe that AI could kill all humans! I personally think it’s >10% within the next decade. I think Anthropic is trying its best, but we don’t yet have a plan to solve the alignment problem for superintelligence and we’re clearly not on track to do so.”
There hasn't been much specialization on how AI might kill us, but the focus has been on the capacity for destructive cyberattacks on utility infrastructure (e.g., water and power plants) and other essential systems on which modern civilization depends.
See also: Gemini Live gains four new features
A major element of the debate is whether the risks can be successfully eliminated with more intensive work on alignment, which means ensuring that the goals of artificial intelligence systems are aligned with those of humanity.
Some argue that this will ultimately eliminate risk by ensuring that AI systems want the same things we do. Others disagree, saying that AI systems may pretend to be aligned to pass security checks while secretly having conflicting goals.
Sayash Kapoor and Arvind Narayanan have become known for tackling what they call “exaggeration, misinformation, and misunderstanding” around artificial intelligence.
The pair have not directly addressed the claim of a >10% risk of annihilation, but have written an extensive essay arguing that it is possible to address the risks of AI through a multi-layered approach to AI security. They contrast the pessimistic view of the AI security community with the more dismissive view of part of the cybersecurity community that this is simply business as usual.
They argue that a more measured middle ground is needed. Specifically, they say that alignment is not sufficient on its own but must be complemented by three additional levels:
“If we act with the right urgency, we can hold AI companies to a higher standard, reduce the risks of losing control, and even tip the attacker-defender balance in cybersecurity back toward the defenders.“
See also: July 17 Android update releases with four fixes for Pixel
While it's uncertain whether AI will kill us by the end of the decade, there are enough reasons to be concerned about AI safety to make it a central focus. The idea that working on alignment is part of the solution but not sufficient on its own seems entirely reasonable.
