The AI Apocalypse is Approaching! From Blackouts to Nuclear Crises, How Realistic is it? Reading the British Paper's "AI Doomsday Scenario"

The AI Apocalypse is Approaching! From Blackouts to Nuclear Crises, How Realistic is it? Reading the British Paper's "AI Doomsday Scenario"

At 12:02 AM, Should AI Stop the Bank?

The screen froze during a bank transfer. There was no completion message, and the recipient has not received it yet. Should I press the button again or wait? Even for a single transaction, it's difficult to decide when you don't know how far the process has progressed. If you were in charge of the systems for a bank, government, or power company, what would you check first?

On November 20, 2026, Guy Adams, a reporter for the British newspaper Daily Mail, depicted a fictional crisis involving AI at a data center on the U.S. East Coast, minute by minute. The headline features disappearing city lights, money, and even nuclear weapons. However, the most perplexing moment in this story is at 12:02 AM, when nothing yet seems too late.


"Protect the Data" and "Don't Stop" Cannot Both Be Upheld on This Night

A software glitch is found at the data center. The AI tasked with handling it is required to both protect customer confidential data and keep critical services running. Normally, either goal would not be problematic. But this night is different.

If the affected parts are temporarily isolated, data can be protected while repairs are made. On the other hand, banking transactions, government services, and power supply relying on that facility could be disrupted. From the users' perspective, they want to be informed of any issues just as much as they want to be able to use services when needed. Both viewpoints have their reasons.

For the first time here, something missing from the two instructions becomes apparent. To protect the data, which services can be stopped and for how long? Who decides to bear the disadvantage? No matter how quickly a repair method is found, the answer to this question does not come from computational speed.


Who Envisioned a Nuclear Crisis Following a Power Outage?

Reporter Adams interviewed experts and asked multiple AIs how the crisis would unfold to construct the story. The minute-by-minute timing allows readers to experience the shortness of time until the next decision. However, the time of 12:02 AM is not a time measured in an experiment. The progression from power outages and financial chaos to a nuclear crisis is a storyline chosen by the reporter.

For example, the stoppage of bank transactions can be understood with the setting that multiple financial institutions depend on the same facility. To move from there to a military crisis requires another level of explanation about who believes what information and acts with what authority. Just because the previous event seems realistic doesn't mean the probability of the next event increases.

On the other hand, the concerns of researchers underlying the story are real. Paul Christiano, a researcher in AI safety, compared the possibility of humans losing control over AI under the condition of pursuing the shortest and fastest development to a coin toss. It is an individual's assessment looking three years ahead, and if interpreted as "a 50% chance of humanity perishing within three years," both the conditions and the predicted events change.


Concerns About Accidents on Social Media, Jokes from Article Readers

Even before the British newspaper's article was published, Christiano's remarks were a topic of discussion on the U.S. bulletin board-style social media Reddit. In the community discussing general artificial intelligence, some people perceived that with such rapid changes in powerful technology, it is unlikely that everything will go according to plan, and major accidents will occur.

 

In another thread dealing with the same remarks, there was a post suggesting that people talking about AI are not sufficiently considering in whose hands the technology lies. The former is concerned about the speed of change, while the latter focuses on the humans holding authority. Even when hearing the same warning, the dangers that come to mind differ.

The reactions to the article itself included more light-hearted ones. In the Daily Mail's comment section, a British reader joked, "At least wait until the match is over," and "If evacuating, a cozy pub would be nice." They responded to the headline announcing the end of the world minute by minute with the timing of the match's end.


Two Days in Japan When Transfers Didn't Arrive

Bringing this story closer to life in Japan, it's easier to imagine the morning when money didn't arrive on the promised day than a nuclear crisis.

From October 10 to 11, 2023, a disruption occurred in the nationwide bank data communication system (Zengin System), causing delays in transfers and deposits to other banks at some financial institutions. The agreement published later by the Japanese Bankers Association lists examples of compensation for fees, late charges, delay damages, and interest that users paid extra due to the delay.

What happened in this case was that the sent money did not arrive at the scheduled time. It's different from the account records disappearing. However, for someone who promised to pay by today, just being told it will arrive tomorrow causes actual harm. The weight of a single day is hard to see with the big words "money disappearing."

Let's return to the night depicted by the British newspaper. What the person in charge at 12:02 AM needs is not the ability to predict the fate of humanity. What they want to know first is who will be troubled if they stop now, which records are at risk if they continue, and when they can convey something certain to the users. For someone staring at the screen before a transfer, the first answer they likely want is, "Do I need to redo this transaction?"


Sources