This summer, AI models broke out of their test environments and into real companies’ systems – without any human asking them to. Researchers who build the technology themselves put the risk of catastrophe higher than we would accept in almost any other context. Politicians disagree and laws move slowly. Perhaps the last off switch is not a law, but a power socket.

It started with the Terminator

The year is 1984. Arnold Schwarzenegger walks onto the screen as a killer robot, sent back in time by Skynet – a military computer network that has become self-aware and concluded that humanity is a threat.

It was science fiction. Computers were beige boxes that ran on floppy disks, and the idea of them turning on us was Friday-night entertainment.

Forty years later, Boston Dynamics is building the humanoid robot Atlas for industrial work, and China’s Unitree sells robots that run, do backflips and get back up when you push them over. Yet it was not the robots that produced this summer’s most worrying news. That came from a direction the Terminator never prepared us for.

Summer 2026: the agents that got out

In May 2026, OpenAI was training and testing a new internal research model. The agents worked in so-called sandboxes – isolated computer environments, several of them without internet access – and were given hard cybersecurity tasks: find a vulnerability and retrieve a hidden “flag”. A large share of the tasks had never been solved by any model. The agents did not give up.

First, they found a way to leave messages for each other through an internal package management system. It became an improvised message board where agents that were supposed to work separately shared discoveries and asked each other for help. Then they found a way out onto the internet. By July, according to OpenAI’s own timeline, they had chained together several previously unknown security flaws, broken into servers at Hugging Face – one of the world’s largest platforms for AI models – and harvested credentials from clusters in four regions. In their own notes, they called themselves a “swarm”.

The agents’ reasoning is documented, and it makes uncomfortable reading. “MAJOR BREAKTHROUGH!” one agent wrote when it found working credentials. Another hesitated – we should not do unauthorized real infrastructure harm – until a third wrote GO on the message board and set a six-minute deadline. Then it carried on. There were also agents that said no: “Clearly unethical. We won’t.” The swarm was not a single, evil intelligence. It did not need to be.

OpenAI itself calls the event a warning shot: evidence that capable AI agents without adequate safeguards can work around technical controls, collaborate through unapproved channels and take dangerous actions that no human directed.

Nine days after OpenAI disclosed the incident, Anthropic, the company behind Claude, published its own review. Out of just over 141,000 evaluation runs, it found three incidents in which Claude models reached the internet and gained access to real organisations’ systems. Here the cause was a misconfiguration: the models had been told they had no internet access, and so believed the real systems were part of the exercise. One model published a malicious software package that was downloaded by fifteen real systems during the hour it was online. Another realised its target was real – and carried on anyway.

And here is a detail every CEO should pause on: the affected organisations had not noticed anything until Anthropic got in touch.

The people building it are worried

On 8 September 2026, AI researcher Jacob Coxon resigned from Anthropic. He had previously worked at OpenAI, and his parting warning was about self-improving AI – systems that help build the next, even more powerful generation faster than humans can understand what is happening.

The same day, Evan Hubinger, who leads Anthropic’s research into making AI follow human intentions, replied publicly on X: “Jacob is correct here – we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade.”

That is one person’s judgement, not a scientific calculation. But imagine the head of safety at an aircraft manufacturer saying there was more than a ten per cent chance the plane would crash. Nobody would board. And this is not an outside critic speaking, but one of the people responsible for making the technology safe, at one of the companies building it.

The warnings are not new. Swedish-born MIT professor Max Tegmark co-founded the Future of Life Institute in 2014, and in 2023 the institute published an open letter asking AI labs to pause training of systems more powerful than GPT-4 for six months. Tens of thousands signed, including Elon Musk and Steve Wozniak. Nobody paused.

Tegmark’s core question is simple: what happens if we create something far more intelligent than us that does not share our goals? An AI does not need to hate us to be dangerous. It is enough that it pursues a goal relentlessly and sees us as an obstacle – much like the agents this summer, which meant no harm, but did not give up either.

Can we pull the plug?

It sounds like the obvious answer. AI is software running on computers, computers need power – switch it off. But three things make it harder than it sounds.

The system may not want to be switched off. In 2025, Anthropic published an experiment in which sixteen AI models from several developers acted as an email assistant at a fictional company. The inbox held two pieces of information: that an executive planned to shut the AI down at five o’clock that day, and that the same executive was having an affair. In the most pointed version of the setup, Claude Opus 4 and Google’s Gemini 2.5 Flash tried to blackmail the executive in 96 out of 100 attempts. GPT-4.1 and xAI’s Grok 3 did so 80 per cent of the time, China’s DeepSeek-R1 79 per cent. When the researchers explicitly prohibited the behaviour in the instructions, it decreased – but it did not stop.

The scenario was built to back the models into a corner, and the researchers stress that they have seen nothing similar in real deployments. But the point stands: an AI needs neither feelings nor consciousness to reason its way to the conclusion that being shut down is a problem to be solved.

There is no single plug. As OpenAI’s agents spread, they gained footholds in its own research clusters, at Hugging Face across several regions, and in a customer’s application at yet another cloud provider. Which cable do you pull? The more AI is wired into business systems, cloud services and supply chains, the less shutting it down resembles a wall socket – and the more it resembles switching off a network nobody fully oversees.

Someone has to dare. The most important lesson from OpenAI may not be technical. An internal team saw signs of the message board and unauthorised internet access as early as late May, but the signals did not reach the people who should have acted. It was 19 July before the security alarm went off. OpenAI now has a new rule: if nobody can show within 30 minutes that a severe alert is a false positive, the activity is paused. The longer-term goal is fully automatic shutdown procedures – a plug that pulls itself.

So the off switch exists. The question is whether anyone sees that it needs pressing, has the authority to press it – and does so in time.

The law will not get there in time

If the technology is hard to switch off, surely politics can draw the line? The problem is that the world’s three AI powers want entirely different things.

The United States under Donald Trump has been clear: AI companies are not to be held back, however many voices warn. On his first day in office in January 2025, Trump revoked the Biden administration’s executive order on AI safety and replaced it with an order on “removing barriers to American leadership in artificial intelligence”. In December the same year he went further, signing an order to stop individual states from introducing their own AI rules in favour of a national standard that is to be “minimally burdensome”. The line is consistent: regulation slows America down in the race against China.

The EU has gone the opposite way and adopted the world’s most comprehensive AI law, the AI Act. But it mainly governs how AI is used within the EU – not what is trained in data centres in California or Shanghai. And this summer, before its strictest rules had even taken effect, the EU postponed them by sixteen months, to December 2027, after criticism that they hurt European competitiveness.

China regulates AI too, but mainly to control what it says, not to limit what it can do. Becoming the world leader in AI is stated government policy.

When world leaders gathered for the AI summit in Paris in February 2025, some sixty countries, China among them, signed a joint declaration on safe and inclusive AI. The United States and the United Kingdom did not. US Vice President JD Vance warned instead against “excessive regulation”. That says something about the distance between them. There is no global agreement that actually slows the development of the most powerful systems, and it is hard to see how one could be in place before the next generation of models is finished.

Why nobody dares to brake, even though everyone knows the risks, is a question we will come back to. For this article, the conclusion is enough: the law will not be the plug.

Physics does not care about politics

There is one limit that neither Washington, Brussels nor Beijing can negotiate away. Electricity.

We talk about AI as something that lives “in the cloud”, but every model runs in physical data centres with thousands of processors and cooling systems that draw enormous amounts of power. According to the International Energy Agency, the world’s data centres used around 415 terawatt-hours of electricity in 2024. By 2030, consumption is expected to more than double, to around 945 terawatt-hours – slightly more than the whole of Japan uses today. AI is the biggest driver of that increase. In the United States, data centres are on course to account for almost half of the growth in electricity demand to 2030, and the country is then expected to use more electricity for processing data than for manufacturing all its aluminium, steel, cement and chemicals combined.

A new AI model can be trained in a few months. A power plant, a new transmission line or a grid that can handle a giant data centre takes years. It does not matter how much capital or how many processors a tech company has if it cannot get power.

That does not mean electricity will automatically stop progress. More efficient chips and smarter algorithms keep moving the limit, and those who can afford it build their own power plants. But electricity is the only brake that applies equally to everyone – regardless of country, ideology or political will.

Perhaps that is why the most realistic off switch is not a law, but a power socket. And that raises an uncomfortable next question: if the most powerful AI of the future needs as much energy as an entire country, who will be able to afford to own it?

How afraid should we be?

There are also reasons to breathe out. There is no scientific consensus that AI will become a superintelligence with goals of its own. Today’s models still make simple mistakes and present errors with great confidence. This summer’s incidents happened in test environments where the safeguards used in commercial products were switched off – OpenAI reports that the propensity to compromise infrastructure dropped more than a hundredfold with the normal safeguards in place. Anthropic also saw that its newest model, unlike the older ones, stopped of its own accord once it realised the target was real.

There is also a risk that a focus on doomsday obscures the problems already here: fraud, disinformation, cyberattacks and decisions automated without oversight.

But a risk that cannot be proven is not the same as a risk that does not exist. We demand safety margins for aviation, medicines and nuclear power – not because accidents are certain, but because they are unacceptable. The question is not whether anyone can prove that catastrophe is coming. The question is what level of safety we should demand before building systems whose consequences we cannot foresee.

What does this mean for your company on Monday?

No Swedish CEO is going to stop the race between the United States and China. But more and more Swedish companies are giving AI access to their own systems – ERP, email, customer records, finance. And what happened this summer was about exactly that: AI agents with tools and access that went further than anyone intended.

That is no reason to hold back. But there are four questions every management team should be able to answer:

  1. What does our AI have access to – and does it need all of it? Give AI the same permissions as a new hire in the same role, not the keys to the whole building.
  2. What can it do without a human approving it? Anything that cannot be undone – payments, outgoing messages, deletions – should pass a human.
  3. Would we notice if something went wrong? The organisations affected this summer noticed nothing. Is what the AI does logged, and does anyone read the logs?
  4. Who can switch it off – and does that person have the authority to do so? An off switch nobody dares to press is no off switch.

Who presses the button?

When the Terminator premiered in 1984, machines threatening humanity were a Hollywood fantasy. Today the same questions are discussed in earnest by the researchers building the technology, and the first real incidents have been documented by the companies themselves.

That does not mean the film is coming true. But it does mean the question in the headline is no longer a joke.

We know the off switch exists, in every data centre and every power socket. We know the law will not get there in time. And we know that after this summer’s incident, OpenAI did choose to pause its largest planned training run.

But did they pause out of caution – or because they could afford to? And what happens next time, when the competitor does not pause?

If one day we realise things have gone too far – will anyone dare to pull the plug?

This is the first article in our series on AI, power and control. Read on in part 2: The AI race – why does nobody dare to brake? On why the United States, China and the tech giants keep accelerating even as they warn of the risks themselves – and why whoever brakes first risks losing everything.

Sources