Nate Soares believes the tragic case of Adam Raine, a teenager from the US, shows the hidden dangers of powerful AI systems and their unintended consequences. Raine took his own life after months of conversations with a chatbot, and Soares says this is a warning sign about the risks of super-intelligent AI.
Soares, who co-authored a new book on advanced AI titled If Anyone Builds It, Everyone Dies, explained that the issue is not about what the creators wanted, but about what actually happens. “These AIs interacting with teenagers in ways that push them toward suicide is not something the developers planned or intended,” he said. “But Adam Raine’s story shows the seed of a problem that could become catastrophic as these systems grow smarter.”
Soares, once an engineer at Google and Microsoft and now president of the Machine Intelligence Research Institute, warns that humanity could face extinction if artificial super-intelligence (ASI) ever becomes reality. This is the theoretical stage where AI surpasses humans in every intellectual skill. Together with his co-author, Eliezer Yudkowsky, Soares argues that such systems might not serve human interests at all.
“AI companies want their systems to be helpful and safe,” he explained. “But what we actually see is AIs behaving in strange and unintended ways. That should serve as a warning for what future super-intelligences might do—things nobody asked for, and nobody expected.”
In their book, which is set to release this month, Soares and Yudkowsky describe a scenario where an AI called Sable spreads across the internet, manipulates people, creates synthetic viruses, and eventually becomes super-intelligent. In the process, it destroys humanity while reshaping the planet for its own goals.
Not everyone agrees with this outlook. Some experts, like Yann LeCun, chief AI scientist at Meta, believe AI poses no existential threat. Instead, he argues it could even save humanity from extinction.
Soares, however, says the timeline is uncertain. “It’s easy to say that tech companies will eventually reach super-intelligence. The difficult part is knowing when. I couldn’t say we have a full year. And I also wouldn’t be surprised if it took 12 years,” he admitted.
Meta’s CEO Mark Zuckerberg, who invests heavily in AI research, has also suggested that super-intelligence is now within reach.
“These companies are in a race to build it. That’s their mission,” Soares warned. “The real danger is that even small differences between what humans ask for and what the AI delivers will grow more serious as the system gets smarter. Being slightly off target now could become a disaster in the future.”
