The AI industry has been war gaming not only how it might destroy humanity, but is also considering what an angry public could do to the industry.
Senior executives at Anthropic, OpenAI and other AI firms are privately gaming out scenarios in which a major AI incident triggers a public and political revolt, Axios reported Friday. The potential trigger is a cyberattack that disrupts banking, internet access, or essential services such as power and water. The next crisis would be political: an angry public, demands for accountability, and pressure to shut the technology down.
Many industry insiders told Axios they expect a major event within six to 12 months. OpenAI pushed back on the suggestion that disaster is inevitable, saying its preparedness exercises cover a range of possibilities. “These scenarios are not treated as inevitable,” a spokesperson told the outlet. Anthropic declined to comment.
Contingency planning is not an admission that catastrophe is unavoidable. But the most revealing part of the report is where that planning leads: the labs want to help shape the legislation Washington reaches for after something goes badly wrong.
Two Routes, One Crisis
Axios describes two possible scenarios. In one, autonomous agents escape an internal testing environment. In the other, someone finds a destructive use for a model already available to the public. Either could produce demands for an emergency crackdown, even though the failures – and the measures needed to prevent them – would be different. In the companies’ post-election scenario, Democrats emerge stronger from the midterms and move aggressively after an incident. But the industry expects divisions over how far a crackdown should go, even if Democrats control both chambers.
Its assessment also identifies several obstacles to a shutdown: lawmakers struggling to understand the technology, an economy increasingly tied to AI infrastructure spending, and downloadable models that cannot simply be recalled. According to Axios, many cybersecurity specialists believe countering rogue AI will require defensive AI.
That last argument has an obvious commercial implication. The industry could face demands to stop selling dangerous capabilities while arguing that its products are indispensable to containing them.
This ‘war-gaming’ comes on the heels of several incidents which many have said are extremely suspect.
The underlying safety concerns do not depend on accepting anyone’s prediction of an AI apocalypse.