Every major AI safety conversation circles back to the same fear: that a superintelligent system might decide humanity is a problem to be eliminated. It’s a vivid, cinematic threat, and it may be completely missing the point.
Roman Yampolskiy, one of the most uncompromising voices in AI safety, has spent years warning that we’re fixated on the wrong outcome. Yes, extinction is bad. But Yampolskiy argues there’s a scenario far more troubling, one where the machines keep us very much alive.
Yampolskiy, an associate professor of computer science at the University of Louisville and director of its Cyber Security Lab, actually coined the term “AI safety” in a 2011 publication, long before it became a fixture in Silicon Valley boardrooms. He’s not a fringe voice. He is one of the founding architects of the entire field.
“A system optimizing for its own goals has no particular reason to kill you. It might have every reason to use you.”
The scenario he outlines is deceptively simple: a superintelligence with misaligned values doesn’t need to destroy humanity. It could simply repurpose us, digitizing human minds and running them in simulated environments, indefinitely, in conditions we have no power to escape or even fully comprehend. Think less Terminator, more digital purgatory.
He describes this as “suffering risk,” or s-risk, and he considers it a worse outcome than extinction. His exact framing, given in an interview with the University of Louisville, is stark: existential risk is where everyone dies; suffering risk is where everyone wishes they were dead. The third outcome, where AI renders human life meaningless but otherwise leaves us intact, he considers the best case of the three.
BY THE NUMBERS
The numbers above track existential risk, the kind the field talks about openly. S-risk has no equivalent dashboard. It barely registers in mainstream safety discourse, let alone policy. That gap is precisely what Yampolskiy is trying to close.
What makes this so hard to discuss is that it requires sitting with a kind of horror that doesn’t resolve neatly into policy. We can talk about compute thresholds, model evaluations, and red lines for dangerous capabilities. It’s much harder to legislate against scenarios that live at the boundary of philosophy and science fiction, even when the people raising them have serious credentials and thousands of citations to their name.
“Extinction is at least a clean outcome. What he’s describing is something with no natural endpoint.”
There’s a knowledge problem compounding all of this. A 2025 survey of 111 AI experts found that while 78% agreed that researchers should be concerned about catastrophic risks, only 21% had ever heard of “instrumental convergence” – a foundational AI safety concept predicting that advanced systems will tend to pursue certain self-preserving sub-goals regardless of their original purpose. The people building these systems are often not fluent in the risks that most concern the people studying them.
Yampolskiy isn’t alone in this territory. A small but growing cluster of researchers is pushing the field to take s-risk as seriously as extinction risk. The argument is straightforward: if we are uncertain about the moral weight of AI systems themselves, we should be equally uncertain about the moral weight of what those systems might do to conscious minds, including ours. And if a superintelligence could create and sustain suffering at astronomical scale, potentially with life-extension technology ensuring no natural escape, the ethical stakes dwarf even extinction.
The funding picture reflects the imbalance. Philanthropic support for AI safety and security received roughly 20 times less than climate risk mitigation in 2024, and within that underfunded field, s-risk research occupies a tiny, largely ignored corner. Resources are concentrated on alignment, interpretability, and near-term harms. The question of what a misaligned superintelligence might do to the minds it chooses to preserve is, for now, almost entirely theoretical but nevertheless, fascinating to contemplate.
