Courtesy of Nate Soares
- Nate Soares said he stopped adding to his 401(k) because AI may kill us within a decade.
- The MIRI president said humanity has a more than 50% chance of dying within a decade.
- Elon Musk said AI could make retirement savings irrelevant by creating an age of abundance.
Saving for retirement is a bet on the future. Nate Soares, the president of the Machine Intelligence Research Institute, isn't so sure AI will let us get that far.
"I do not save for retirement right now. I have a 401(k) from when I worked at Google, but I haven't contributed for a decade," Soares told The Atlantic CEO Nicholas Thompson in a recent interview for Thompson's "The Most Interesting Thing in AI" series.
Soares' decision reflects a grim view of the AI race.
"Godfather of AI" Geoffrey Hinton and Anthropic researcher Evan Hubinger have, respectively, put the odds of AI killing everyone within a decade at 10% and more than 10%. Soares is more pessimistic: He told Thompson he believes there is a greater-than-50% chance AI causes human extinction in less than a decade.
Soares, who is also the coauthor of "If Anyone Builds It, Everyone Dies," has spent more than 12 years working on AI alignment, the field focused on ensuring powerful systems act as intended. Before joining MIRI, the Berkeley-based nonprofit focused on AI risks, he worked as an engineer at Google and Microsoft.
Soares said that a sufficiently capable AI could develop goals at odds with humanity's, evade oversight, improve itself, and build the automated infrastructure needed to operate without people.
From there, he said, it could reshape the planet around its own objectives, potentially making it uninhabitable for humans — a warning he shared with Business Insider last year.
A 401(k) for the apocalypse?
The question of how AI will shape the future has also prompted Elon Musk to question whether retirement savings will matter. Musk's vision of the future, however, is radically different than Soares'.
In January, the Tesla and SpaceX CEO said people should not worry about "squirreling money away for retirement" in 10 or 20 years, predicting advances in AI, robotics, and energy will make retirement savings "irrelevant." (Personal finance experts and AI gurus told Business Insider the opposite: You really should be saving for retirement.)
Musk has said that supercharged productivity could create an age of abundance, in which everyone has access to a "universal high income," cheap goods and services, and better healthcare and education.
Musk has also said money could become irrelevant in a future where AI and robots outperform humans at virtually every task.
For Soares, the prospect of a vastly capable AI is an existential danger, not a route to plenty.
Echoing calls from tech leaders to "pace the frontier" of AI development, Soares said governments should halt the training of general-purpose AI systems. They should focus on developing technology for applications like cancer research and safer self-driving cars, he said.
The solution, he told Thompson on Wednesday, "is to not build the thing that would kill everybody."
"I do everything I can to stop this eventuality," he said. "I don't think it's inevitable."