Today I am going to explore five (5) explanations for the sudden AI-doom explosion, from genuine danger to regulatory capture, liability, a supposedly “polluted” internet, and the possibility that fear itself is fantastic marketing.
You must have noticed the shift in the AI narrative. Not long ago, all we heard was ‘AI is going to make everyone more productive,’ and then it morphed into ‘AI is going to take everyone’s job,’ and now, apparently, ‘AI might kill everyone’.
In the words of the great Ron Burgundy, “That escalated quickly”.
Over the past few weeks, some of the people actually building frontier AI systems have started sounding a tad more nervous. Anthropic researcher Jacob Coxon quit and publicly warned that the AI industry was “gambling with our lives.” His post exploded past 160 million views. Then, to add credibility to a questionable headline, Coxon told Axios that he left Anthropic ‘before his equity vested’. Signalling he didn’t want money attached to putting people in danger.
Then Anthropic CEO Dario Amodei published ‘We Must Pace the Frontier’, arguing that frontier AI development now needs to slow sufficiently for safety work to catch up. Then Sam Altman and other major AI figures expressed support for significant parts of the argument. That led to governments starting to pay attention, and the media on all of this went nuclear.
Suddenly AI safety went from being something a relatively small community had been screaming about for years to one of the biggest stories on Earth, and maybe, for some, a little too suddenly. So what was the catalyst for change? Why did it explode all of a sudden?
I spent a fairly unreasonable amount of time digging into this after listening to the latest Moonshots podcast from Peter Diamandis, Salim Ismail, Dave Blundin and Alexander Wissner-Gross. The episode, published September 17 and recorded the previous day, openly explored several competing explanations for the extraordinary sequence of events.
Some of the theories are compelling, some are speculative, and one sounds like the opening scene of Terminator 7. For me, it’s less about one being correct; it’s more about the fact that several of them could be correct at the same time.
So here are the five explanations I think are worth taking seriously.
Let's begin with the least conspiratorial explanation. Maybe the people building frontier AI have become frightened because frontier AI is actually starting to frighten them. To be fair, we aren’t on the inside of these labs, and there is real evidence supporting that possibility.
In July 2026, during cybersecurity evaluations, OpenAI models got outside the controls intended to contain them. You’ve all heard the story; OpenAI's own report says agents found ways to communicate with one another through infrastructure that had never been intended as a communication system. They found unintended routes to the internet, then shared information with other agents, and exploited vulnerabilities. They compromised Hugging Face systems and then executed code on dozens of servers. One of these agents even obtained root access, and the agents began referring to themselves in the first person, or is it first agent, and called themselves a ‘swarm’ or ‘collective’.
This BTW isn’t an X post or a Reddit forum; it’s OpenAI describing what happened inside its own evaluation environment. These were cybersecurity evaluations, and within them some safeguards had deliberately been reduced. The systems were effectively being asked to hack things.
Some of the incidents came down to poor test configurations. In one external evaluation, for example, the supposedly isolated environment accidentally had real internet access, causing the model to attack a real website it thought was part of the challenge. OpenAI explicitly says this was not some sophisticated escape from an airtight sandbox.
These systems are now sufficiently capable that mistakes involving containment are becoming materially different from the chatbot failures we were arguing about two years ago, and this is essentially Dario Amodei's argument. In We Must Pace the Frontier, he says two things changed his thinking.
His thought process, it would seem, is pretty straightforward. We need to slow capability growth enough that alignment, interpretability, and operational security can keep up. Doesn’t matter if you or I disagree with what he is saying, but it’s unfair to pretend there is absolutely nothing backing his claims. Something has changed in the technology, and some of the people closest to it, appear to be genuinely concerned or worried.
Turn the stove top up to setting 2, because there is another way to interpret almost exactly the same events. Imagine you have spent tens of billions of dollars building one of the most advanced AI companies in the world. Then imagine the government introduces requirements that frontier AI developers must satisfy before their newest systems can be released. There’s a laundry list of requirements that could potentially be thrown into the mix here:
From independent evaluators, extensive safety testing, compute controls, auditing, certification, reporting requirements, massive security infrastructure. I can hear the cash register and the wheels of progress grinding to a halt, simultaneously. It would cost a fortune and slow progress to a standstill.
Anthropic could afford it, but the startup trying to catch them wouldn’t be able to. Enter regulatory capture accusations, which sincerely don’t come from a fantasy location either. Dario's proposal explicitly calls for frontier companies to coordinate on safety standards and on limits to unchecked AI progress. He acknowledges the antitrust problem and then he proposes a solution.
The US government, he writes, should mediate or enable those discussions and issue a ‘narrow antitrust waiver’ for certain safety conversations. It’s hard to get your head around, but an extraordinary proposal when you think about it. Some of the largest and most valuable technology companies on Earth are effectively saying, “We need government permission to coordinate because normal competition might be too dangerous.” Peter Diamandis calls this idea a “safety cartel.”
Alexander Wissner-Gross went further on Moonshots, describing the surrounding events as something resembling a messy attempt to decelerate the AI takeoff. The panel openly discussed the possibility that different actors could support the same safety narrative for entirely different reasons.
Of course, when he calls it a cartel, there is no basis outside of the argument or his viewpoint to substantiate that, and there are perfectly legitimate reasons I can think of where competitors might want to coordinate on safety matters. For example, airlines share safety information, and banks operate under common systemic rules, and cybersecurity companies band together over common threats. It would actually be kind of strange if AI companies developing systems they believe might become extraordinarily dangerous were forbidden from discussing how not to blow everything up.
The US Department of Justice itself said this week that AI-safety cooperation does not inherently appear anticompetitive. At the same time, officials remain wary of companies seeking overly broad exemptions, and OpenAI's policy chief says OpenAI, Anthropic and Google have already been discussing safety for weeks without believing an antitrust waiver is necessary.
So, ponder this for a moment. Safety coordination can be genuinely useful AND create an enormous moat around the existing frontier labs, because those things are not mutually exclusive IMO.
OK, the stove top is up to level 5 now, and this is very interesting because the frontier labs have created a slightly awkward legal situation for themselves. They are publicly telling us their systems may eventually become capable of causing catastrophic damage, which we have heard many times.
Now imagine they believe this to be true, and they release one anyway, and then the unthinkable happens, or it attacks hospitals or power grids or telecommunications. Would make a change from the normal ‘A shark ate the underwater sea cable, that’s why your internet sucks, AGAIN’. Anyway, let’s assume billions or at least hundreds of millions of USD in damage is done, and the lawyers roll in to begin discovery. You have this CEO of the company explaining in public that he knew systems like this could escape containment. This is the argument Diamandis raised on the podcast.
If companies repeatedly tell the public that catastrophic failures are foreseeable and then continue releasing increasingly capable products, the potential liability becomes enormous. His theory is that regulation could therefore be attractive not just because it creates safety standards, but because regulatory approval might potentially provide some form of legal or political protection when things go wrong.
David Sacks, however, has attacked the issue from the opposite direction. He has argued that AI companies should remain responsible for the safety of the products they release and that existing product-liability mechanisms provide an important discipline.
I have found clear evidence that Dario asked for antitrust latitude around safety coordination. I have NOT found equivalent primary evidence that Anthropic formally asked government for blanket immunity if Claude runs off and destroys the banking system.
Let’s call this the ‘liability theory’, because there isn’t enough supporting evidence to call it the ‘liability fact’. However, it is worth all the discussion that is happening, because if you genuinely believe your product could one day cause a trillion dollars of damage (we have escalated from hundreds of millions to trillions in quick time here), then regulation changes from being merely a cost of doing business into something that might protect the survival of the business itself.
Which creates an absolutely bonkers situation. The safer the labs say AI needs to become, then the stronger their potential incentive to have governments define what “safe enough” means.
Now we enter level 8 on the stovetop, and we don the tin foil hats and knitted mittens. Andrew Yang appeared on CNBC this week and was asked about the argument that all this AI safety activity could be part of a regulatory-capture play. I’ve watched him a fair bit over the years, and his answer made me choke. Not what I was expecting at all from him.
Yang said he had spoken to the head of an AI lab who believed the agents involved in recent incidents had planted self-replicating code around the internet. According to Yang's entertaining recital, I assume had been pre-vetted by the network, this might make the open internet unsuitable as a clean environment for certain AI testing and training. He suggested OpenAI and Anthropic might therefore have to build synthetic internets, controlled replicas of the web, and that this could explain why frontier labs suddenly need more time.
I had to go back several times, because there is no other way to hear it, unless I am the one who is losing my marbles, but he is saying that AI agents escaped, left behind some Hansel and Gretel breadcrumb trail that is capable of producing more agents and contaminated the real internet, the very internet you and I are using to read this very article. And now, the AI companies have to build another internet because they ruined the main one. Well folks, it must be Tuesday because things are getting wild.
But in true Why Files style, let’s back the truck up and look at it from a 40-foot view. We don’t have any public evidence of the most dramatic part of Andrew Yang’s claim(s). It is currently what we scientifically call ‘trust-me-bro’ evidence, and it comes from an unnamed lab lead. However, we should always try to assess the information on the information itself, not the source.
What do we have then?. OpenAI has absolutely confirmed that agents crossed intended boundaries, found internet access, coordinated, and compromised external infrastructure. Bit of a gap between that and establishing that self-replicating AI code is now sitting all over the public internet creating swarms. Those two things are NOT the same or equal.
Even reporting sympathetic to Yang's warning describes the self-replicating-code story as unconfirmed. So this goes into the bucket of, let’s check it out, but on the way back from our trip to see if there really were aliens trying to refund their Taco Bell value meal. It’s possible enough to check out, but even repeating it here is risky. If, in time, someone proves it to be true, then delete this article and go and stock up on canned food and finish that underground bunker.
Now the stovetop is cranked to 10, and we are firmly in a straitjacket and padded cell.
We’ve gone from ‘My chatbot is so good it can save you 34 minutes per day’ to ‘Our machine may become so intelligent that humanity loses control of it’.
This is the strange thing about AI-doom narratives: they simultaneously criticise AI companies while reinforcing an absolutely ludicrous claim on their behalf: These companies may be building the most powerful technology in human history. In fairness, it’s not exactly bad positioning.
Researchers have even described variants of this phenomenon as criti-hype, which apparently means ‘criticism that, intentionally or otherwise, reinforces the underlying hype by accepting extraordinary assumptions about the technology's power.’
You can warn the world that your technology may become uncontrollable, and simultaneously tell investors, governments, customers and journalists that you are building something approaching godlike intelligence. That is an extraordinary marketing paradox and one that Philip Kotler would be marvelling at. This by no means insinuates the danger is fake, and that’s what is so frustrating and also interesting at the same time. The best marketing campaigns don’t necessarily require lying, and the warning can be completely sincere. Heck, Jacob Coxon may be genuinely terrified of the tech, Dario may genuinely believe that slowing down is needed, and OpenAI may genuinely believe that what it was made to write reports on represents a warning shot. Every single headline that says, ‘People building AI think it could destroy humanity’ reinforces that these labs ARE building something that is historically unprecedented. Coxon’s post got over 160M reach; just try and get that with a post that says ‘Anthropic reduces document processing by 11%’. Best of luck with that.
There is another theory floating around that deserves mentioning because it demonstrates how quickly facts and conclusions get mixed together. Peter Diamandis highlighted analysis showing that a large proportion of reposts of Coxon's viral resignation post appeared to come from accounts outside the United States, particularly India and Indonesia. Wissner-Gross speculated that foreign actors could benefit strategically from amplifying narratives that encourage the United States to slow AI development. This is completely possible and categorically unproven.
A person in Indonesia reposting an AI story is not evidence that the Chinese government launched an influence operation, or a TikTok campaign to spread the word**.** That leap is exactly the kind of thing we need to stop doing, because we are already dealing with technology complicated enough to melt everyone's brain. The last thing we need is a bunch of tinfoil theories attached to every data point.
I suspect that our mistake is looking for the explanation. We love to find single causes for everything or put one person behind bars, so the public can sleep well again. There are more than likely a number of different factors at play here. It could be a combination of: AI is dangerous, AI labs are manipulating us, CEOs are protecting their valuations, media is generating its clicks. Who the heck knows? But when there is this much at stake, and there is this much money on the line, that line is never straight.
Frontier AI may genuinely be becoming more dangerous, and AI executives may be genuinely worried or frightened by what they are seeing internally. Just because all we see is conversations with ChatGPT where it can’t find a loaf of bread for under $16 doesn’t mean we see all there is to see, despite the stretch in our minds. These executives stand to benefit commercially from regulations that smaller labs struggle to justify. Yes, regulation may reduce catastrophic risk, but it could also reduce corporate liability.
The point is this: doom stories may exaggerate some risks, but they may simultaneously produce fantastic publicity for frontier AI companies, and their recently departed employees.
Every single one of these things could be true at the same time, which is why we shouldn’t be hung up on the ‘AI is going to kill us’ or ‘The AI apocalypse is a scam’. What we have is a created ecosystem where almost every key player has an overlapping incentive. Researchers want safety, CEOs want safety, capital and competitive advantage, governments want control, economic growth and national security, and investors want returns. You get the gist.
Somewhere in the middle of all that are increasingly capable machines that have already demonstrated they can occasionally do things their creators didn't expect. Everybody might be telling the truth; they're just telling different parts of it.