There have been dire warnings lately about the threat to humanity from AI. One Anthropic researcher claimed there was a 1-in-10 chance of an AI driven apocalypse in the coming decade, citing the threat from AI with superhuman intelligence [1] with another even claiming the odds may be 50%+ [1a]. So how might this apocalypse play out?
Personally, I am sceptical of the threat of a superhuman AI deciding to kill off or enslave the human race. If an AI system is really so intelligent, it would realise that the data centres and electricity grids that are its nervous system and lifeblood will be dependent on humans, and are likely to be for some time, notwithstanding advances in robotics [1b]. Assuming it has a bias towards self-preservation, it wouldn’t seek to wipe out the human race. My fear is more from mistakes caused by AI systems, and also from AI being mis-used by humans with genocidal intent.
Reverse stress testing where an AI event threatens the human race,...
