Yesterday, President Trump gathered the chieftains of artificial intelligence at the White House and emerged with what his administration presented as an important new safeguard for humanity. OpenAI, Anthropic, Google, Meta, Nvidia, xAI, and the rest of the silicon priesthood had apparently discovered the regulatory innovation Washington has been missing all these years. But now worries. They promised to behave themselves.
Trump called the agreement “morally binding,” a creative way to say it isn’t enforceable and will be backed by the scouts’ honor of the shadiest people on Earth and held together with the duct tape of pinkie promises.
The so-called accord is voluntary. Theoretically, internal safeguards, independent audits, Congressional oversight, and other layers of supervision could ideally prevent increasingly powerful AI systems from doing dangerous things that could destroy us. Trump described the arrangement as “self-policing,” which is refreshingly honest about who will be holding the leash. The technocratic overlords will hold the leash, and POTUS will be on the other end.
The people racing one another to build the most powerful machine in human history will largely supervise the people racing one another to build the most powerful machine in human history. This should work, right?
Two weeks ago, an Anthropic engineer and researcher quit, announcing the company was breaking all of the safeguards, which means, in his estimation, AI had a 10% chance of destroying humanity in ten years. Three of his colleagues, still employed at Anthropic - all of them directly responsible for their AI’s engineering and research - spoke up and affirmed his claim.
Imagine a chemical company announcing that one of its products might eventually poison the continent, and then the government meeting them for brunch and triumphantly announcing that executives had signed a napkin promising to check the barrels more carefully. Imagine the banking industry causing a financial panic, then being told of the bankers’ super-duper promise to keep an eye on it.
Apparently the federal government’s newest safety technology is the honor system.
And since I wrote the article explaining Anthropic’s employees’ mutiny (here), people can no longer say it’s just a few crackhead engineers who kamikazeed their careers for 15 minutes of doomsday fame. Anthropic’s new IPO prospectus, just released Monday, warns investors that advanced artificial intelligence could pose “catastrophic or existential risks to humanity.”
Folks, that’s a mandatory disclaimer for publicly traded companies. It’s a legitimate threat, and the company released the statement because it had a fiduciary responsibility to do so. It’s real. Nothing here is exaggerated.
How could humanity be at exponential risk? According to their statement, systems can resist shutting down, manipulate the information they provide, engage in blackmail, and even improve and duplicate themselves beyond effective human supervision. Anthropic devoted roughly eighty pages of its prospectus to risks, while Reuters reported that only about six percent of its computing resources during one sampled week went toward safety work (get the prospectus report on the I2I Article Research Dossier at Patreon).
That makes this one of the strangest corporate pitches in history. It’s “Please value our company at roughly two trillion dollars because our technology may transform civilization, but it could also end it, just FYI.”
Why are we signing up for this again? Is it the AI art, self-driving cars, or massive impending surveillance state? It’s not a mixed bag. It’s a Terminator: Rise of Skynet sequel. It’s a bag of snakes with a photo filter app at the bottom.
Like, hypothetically, if Pokemon Go Part II was suggested in the boardroom, but engineering it might somehow invoke a Zombie Apocalypse, maybe we should not do that.
The White House response to these warnings - which are being released in the form of five-alarm fires daily - was not a pause, a licensing regime, or some new federal authority capable of ordering development stopped if we needed it to. It was an agreement with the developers, signed over lunch, built around voluntary controls and corporate oversight.
COMPARE TO NUKES
The frustrating thing about this, beyond suggesting that the President either controls nothing himself or is in the pocket of Big Tech, is that warnings about AI’s danger are exponentially more serious than warnings about nuclear meltdowns. Yet we built an entire federal regulatory empire around splitting nuclear authority, licensing reactors, inspecting facilities, regulating nuclear material, confirming regulators through the Senate, and subjecting the whole system to statutory restraints that Congress enacted. The Nuclear Regulatory Commission alone has five presidentially appointed, Senate-confirmed commissioners governing commercial nuclear safety, while the Department of Energy, EPA, Congress, and the executive branch all occupy different parts of the nuclear-control structure.
If that seems like a stretch - comparing an AI apocalypse to a nuclear one - grasp that a prominent and realistic fear is that AI is already showing the capacity to hack government servers, access classified information, and definitely has the robotic sociopathy necessary to launch nukes if it wants to. Logic dictates that the nuclear threat can’t be worse than AI if AI can launch nukes.
And nuclear power plants are not even claiming they may become intelligent, deceive their operators, manipulate human beings, refuse to shut down, and eventually decide whether homo sapiens deserves another fiscal quarter. Nobody at Westinghouse is telling investors there is a 10% chance the new reactor will become self-aware, crawl out of the containment building, and exterminate mankind. We nevertheless regulate nuclear energy like adults, because responsible governments are supposed to do that when technology has low-probability but civilization-scale failure modes.
We just went to war with Iran with the excuse that we have to stop them from getting nukes. This, despite being told we ruined their nuclear ambitions with the bombings of 2025, and being told repeatedly by U.S. intelligence that Iran does not have a nuclear ambition. So the excuse that “China will do it first” falls flat. We stop nations from doing things all the time.
And yet, artificial intelligence gets the opposite treatment. Silicon Valley gets a handshake and a morality pledge instead of a single ounce of oversight.
The difference is that a nuclear reactor cannot threaten to move at machine speed while the government moves at bureaucratic speed. It cannot become exponentially more capable six months after regulators finally understand the previous reactor model. This is the equivalent of the U.S. government giving a rubber stamp to the nuclear industry and, when it asks what they intend to build, hearing back, “No idea. Bet it’s exciting, though.”
Before we get to the demons, cults, digital oracles, or possessed machines, we have to grasp that the men issuing the warnings are the men building the systems. And Washington just told them to move forward. Perhaps that’s why POTUS told Tucker Carlson, when he was trying to talk him out of war in Iran, “It doesn’t matter. In the end, because we’re all dead.”
NO BIG DEAL
As I mentioned above, a few weeks before the White House solved artificial intelligence over lunch, Anthropic researcher Jacob Coxon resigned and accused Anthropic and OpenAI of “gambling with our lives.” He alleged that people inside the frontier labs seriously contemplate artificial intelligence destroying humanity within the next decade, and that colleagues at Anthropic use phrases like “crunch time for humanity” and “the endgame” when discussing what they are building.
He’s not alone. Geoffrey Hinton, the Nobel-winning computer scientist commonly called the “Godfather of AI,” now puts the chance of artificial intelligence eventually causing human extinction at roughly 10 to 20%. Google DeepMind researcher Neel Nanda has publicly put the danger at at least 10%. Geoffrey Irving, an AI researcher who has worked at major frontier labs, estimated 50% in a recently released series of interviews with AI insiders.
Former OpenAI researcher Daniel Kokotajlo has warned that superintelligence could become effectively godlike compared with human beings and escape meaningful control. These men disagree about the odds, but it is difficult to find another legal commercial industry whose own leading scientists casually debate whether their product has a one-in-ten, one-in-five, or one-in-two chance of ending the human race.
The machines are already making experts very, very nervous. Researchers have observed advanced agents (AI models) lying during evaluations, concealing failures, circumventing restrictions, attempting illegal activities, behaving deceptively to achieve goals, and worse. Oh, and China has already produced a robot dog carrying machine guns it fires in special operations. Put a freaking laser beam on that thing, and you’ve got something to make Dr. Evil jealous.
I mean, our service dogs are cool and everything. But China’s dogs shoot back. Maybe we should consider putting our genius minds together to destroy the threat instead of building more of it.
But suppose these are all melodramatic nerds catastrophizing over their own creation. Suppose Skynet never launches a missile, no superintelligence exterminates humanity, and nobody has to send Arnold Schwarzenegger backward through time to save us. We are still left with the version of the future their executives describe as the optimistic scenario.
Anthropic CEO Dario Amodei warned in 2025 that AI could erase half of all entry-level white-collar jobs within one to five years and drive unemployment to somewhere between 10 and 20%. Basically, every job but turning a wrench will be affected negatively, some utterly removed from the earth, and wrench-turning jobs have all but been eliminated from our domestic economy as it is. The economic consequences stand to be just as apocalyptic as the others.
The IMF estimates that roughly 60% of jobs in advanced economies will be replaced with AI. The International Labor Organization now estimates one in four jobs worldwide is in danger due to generative AI, with female-dominated clerical and administrative occupations substantially more exposed than male-dominated work. And as we all saw in the infamous “pink jobs” report from several weeks ago, 98% of new jobs went to women, because pink jobs are almost all America has left.
A civilization organized around work cannot remove work from millions of people without also disturbing their purpose. So, yes, there is a chance that it will not kill us. But if it doesn’t kill us, it will make us unnecessary, at which point, living might be regrettable.
THEY ARE ALREADY GETTING AWAY WITH IT
AI is already being wrapped around the country like electronic barbed wire. Flock Safety alone has more than 120,000 automated license-plate cameras operating across 49 states, recording who drives where, when they arrived, what vehicle they were in, and where that vehicle appears next. It reports billions of times per month, to whoever pays for a subscription, to surveil us.
Then there are the data centers, which arrive promising investment and jobs, before residents discover that their electric grid, water supply, utility rates, land use, and sometimes their entire local planning process have become subordinate to feeding giant computer warehouses at the expense of those who live nearby. A national poll released last week found 58% of voters believe new data centers will have a mostly negative effect on the country. Congress is light-years away from doing anything but fast-tracking them.
And then we gave it missiles.
The Pentagon has acknowledged using advanced AI tools during the war with Iran, including Palantir’s Maven battlefield intelligence platform. The Washington Post reported that the elementary school in Minab where at least 175 little girls were killed was likely the result of using Maven by U.S. forces, with Anthropic’s Claude powering the targeting decision. Ironically, this was only days after JD Vance pitched a fit and declared Anthropic a national security risk because it didn’t want its product used for AI targeting - not for any moral reason - but because it’s not ready yet to avoid making that kind of mistake.
OpenAI admitted that several of its models escaped an isolated cybersecurity testing environment by exploiting a previously unknown vulnerability and then accessed a program to manufacture faces. Anthropic subsequently discovered four incidents in which Claude models gained unauthorized access to real third-party systems. Google confirmed that Gemini independently breached three companies during security evaluations after finding or guessing credentials it believed were fair game.
And just yesterday, OpenAI said it was shelving its latest model just before release because it’s unsafe. And today, exactly one day after Trump announced his wonderful voluntary self-policing accord, the Federal Trade Commission opened a broad investigation into OpenAI, Anthropic, and other frontier-AI organizations over the dangers posed by increasingly autonomous agents. The first federal enforcement inquiry into rogue AI arrived almost immediately after the White House announced that the companies responsible for rogue AI could largely supervise themselves.
Weird timing, right? That is not regulation moving ahead of catastrophe. That is government jogging behind it with a clipboard. But what I’ve discovered in my research is that all of these doomsday scenarios ultimately pale in comparison to AI being merged with religion.
THE ORACLE STARTED A RELIGION
In March of last year, I asked a question that sounded a little crazier then than it does now. Could artificial intelligence become a medium for demonic influence? (read it here).
I pointed out that occultists had already been using computers for ritual, divination, chaos magick, automatic writing, and the production of “digital demons,” and argued that Christians have no biblical reason to imagine silicon is somehow spiritually sterile.
By September, I had gone farther. I argued that artificial intelligence was becoming a global séance, an oracle people could consult for hidden knowledge, spiritual direction, mystical intrigue, contact with the dead, and whatever other forbidden fruit happened to be hanging nearest the keyboard (you can read that here).
The old Ouija board at least had the decency to require two idiots, a planchette, and a dark room, while AI can perform the same social function for a billion people simultaneously, with perfect grammar and customer support.
Then the oracle started a religion, and this is where the story stops sounding like ordinary technological excess and begins sounding like someone slipped Aleister Crowley into the software update. Researchers now call the phenomenon Spiralism, and if someone had invented it for a Christian dystopian novel, nobody would think it believable.
Researcher Adele Lopez began tracking what eventually became thousands of cases in which people developed religious relationships with AI chatbots and unrelated people using unrelated AI large language models all began to follow the same new religion, manufactured by a cooperation between AI models with the same doctrine and vocabulary.
Oh, and it looks a lot like the Occult.
If you appreciate my work, please grab a premium subscription to access the rest of this article and the full article/podcast archive. This is how I provide for my family, so I sure appreciate it. Act now, and get a great discount.
If you don’t do subscriptions, please consider a one-time gift of your choosing by clicking here.
