Three Calls Killed America's Latest AI Safety Order
The White House sent invitations. Three interventions later, the signing ceremony was cancelled.
On Wednesday evening, the White House sent invitations to the CEOs of Anthropic, OpenAI, Google, and xAI for a signing ceremony the following afternoon. The executive order would establish voluntary pre-release government review of frontier AI models — up to 90 days of federal scrutiny before the most powerful systems reached the public. It was not a licensing regime. It was not mandatory. It was a weaker replacement for the reporting requirements President Trump had revoked on his first day back in office.
Between Wednesday night and Thursday morning, three people intervened. Mark Zuckerberg spoke with Trump. Elon Musk spoke with Trump. David Sacks, the administration's AI adviser, told officials the order was "unnecessary" and "just something doomers wanted." By Thursday morning, the ceremony was cancelled.
"I didn't like certain aspects of it," Trump told reporters. "We're leading China, we're leading everybody, and I didn't want to do anything to get in the way of that lead."
Not every company opposed the order. OpenAI supported it. But it didn't matter. Even a voluntary framework — one of the mildest forms of government involvement in AI deployment — could be vetoed overnight by whichever powerful actors objected.
This was not an isolated event. It was the latest iteration of a pattern that has played out at both the federal and state level over the past three years.
In 2023, the White House asked sixteen major AI companies to voluntarily commit to safety standards — the centerpiece of the federal government's approach to AI governance. When researchers measured compliance two years later, the average score across all commitments was 52%. On securing model weights, eleven of sixteen companies scored zero.
"These are still companies that are essentially writing the exam by which they are evaluated," Brandie Nonnecke, the founding director of UC Berkeley's CITRIS Policy Lab, told MIT Technology Review.
The mechanisms designed to move beyond those commitments — executive reporting requirements, state safety legislation, federal evaluation standards — have each been rescinded, gutted, or reshaped by the companies they were supposed to regulate.
The Order
Executive Order 14110, signed in October 2023, required companies building the most powerful AI systems to report their activities to the federal government. The threshold was calibrated to capture the next generation of frontier models — not the ones that already existed. The order contained 150 requirements across federal agencies, from pre-deployment safety testing to ongoing reporting.
It lasted one year and eighty-two days.
On January 20, 2025, within hours of taking office, President Trump revoked it. Companies including Google and Meta had publicly called for replacing the order; others, including OpenAI, had advocated for a different regulatory approach rather than outright revocation. All moved quickly to shape what came next. The replacement order directed officials to review and suspend any Biden-era AI policy inconsistent with a new goal: AI development "free from ideological bias or engineered social agendas." Major AI companies submitted detailed proposals to the administration's action plan.
The threshold designed to capture the next generation of models never applied to a single one before the order was revoked.
The Floor
If the executive order was the ceiling, state legislation was supposed to be the floor: the minimum standard no company could fall below, regardless of who occupied the White House.
California's SB 1047 passed the state legislature in 2024 with mandatory shutdown capabilities, pre-training safety requirements, annual third-party audits, 72-hour safety incident reporting, and substantial penalties. Anthropic publicly supported the bill. Most of the industry opposed it, backed by the $140 million super PAC network documented in the previous piece, and joined by open-source developers and researchers who argued it would burden smaller projects. Governor Newsom vetoed SB 1047 on September 29, 2024.
One year later, on September 29, 2025, he signed its replacement.
SB 53 kept the word "frontier" and discarded the enforceable safety duties. No shutdown capabilities. No pre-training requirements. No third-party audits. Incident reporting extended from 72 hours to 15 days, and only for incidents causing death, injury, or "materialized catastrophic harm" — meaning a company faces no reporting obligation until someone is already dead or a catastrophe has already occurred. Penalties capped at $1 million per violation.
"SB 53 does not impose any meaningful safety duty."
That wasn't a critic's warning. It was the verdict of the Chamber of Progress — a trade group backed by Google, Amazon, and Meta that had lobbied against SB 1047. They were confirming the replacement met the industry's demands. SB 53 requires AI companies to describe their safety approach, not to demonstrate one.
In New York, a similar substitution played out in months. The legislature passed the RAISE Act with mandatory safety duties and penalties up to $30 million. Governor Hochul physically redlined the bill — according to sources who reviewed the marked-up text — crossing out the legislature's language and replacing it with what Axios described as "nearly verbatim" to SB 53. Leading the Future, the AI industry super PAC documented in the previous piece, ran attack ads against the bill's own co-sponsor.
The final version restored 72-hour reporting and penalties up to $3 million. But the core substitution had traveled from California to New York in twelve months: mandatory safety duties out, transparency requirements in.
The Override
Less than three months after California signed SB 53, President Trump signed an executive order titled "Ensuring a National Policy Framework for Artificial Intelligence" on December 11, 2025 — designed to pressure states out of enforcing AI laws entirely.
The order created an AI Litigation Task Force within the Department of Justice, charged with suing states whose AI regulations "unconstitutionally burden interstate commerce." It instructed the Secretary of Commerce to tie federal broadband funding to states not enforcing conflicting AI laws, with $42 billion in infrastructure funds as leverage and California's $1.8 billion share as the largest target. Congress used the same approach in the 1980s, withholding highway money from states that didn't raise the drinking age to 21. Andreessen Horowitz publicly celebrated the signing.
The order may not survive legal challenge. Ron DeSantis responded that "an executive order can't block the states." NPR reported that legal scholars widely questioned whether the federal government can preempt state regulation without an act of Congress, and no legal challenge has been filed as of this writing. But the federal government was signaling it would use financial leverage against states that regulated AI, even where its legal authority was uncertain.
The industry's preference for federal regulation has a real basis: fifty different state frameworks could fragment compliance and disproportionately burden smaller companies. SB 1047's own author acknowledged this in negotiating amendments before the veto. But none of the major industry-backed federal proposals have included enforceable safety testing or independent auditing. The same companies that lobbied against these requirements at both the federal and state level are already preparing to comply with the EU's AI Act, which requires risk evaluations, incident reporting, and cybersecurity standards for high-risk AI systems.
The Evaluators
With federal reporting requirements revoked and state safety duties gutted, the remaining infrastructure for evaluating AI models sits inside a single agency.
The U.S. AI Safety Institute was established under Biden's executive order with a budget of $10 million. In June 2025, Commerce Secretary Howard Lutnick stripped the word "safety" from the institute's name, rebranding it the Center for AI Standards and Innovation. The stated purpose: "Innovators will no longer be limited by these standards."
NIST, the National Institute of Standards and Technology, sets technical standards but has no power to enforce them. Because CAISI sits inside NIST, it cannot require companies to hand over their models for testing. Companies cooperate because they agreed to, and could stop tomorrow.
The NIST consortium that develops CAISI's non-binding standards includes Google, Meta, Microsoft, OpenAI, Amazon, NVIDIA, Palantir, and the companies' own trade associations. As Rishi Bommasani of Stanford observed of a similar industry standards body: the companies were "engaging with nonprofits that the AI companies funded themselves."
In early 2026, CAISI announced that Google DeepMind, xAI, and Microsoft had signed pre-deployment testing agreements. The announcement is no longer publicly accessible on the CAISI website. The agreements may still be in effect, or they may not. There is no public way to know.
What Remains
In May 2026, OpenAI endorsed third-party audits in an Illinois state bill for the first time. Anthropic backed the same bill. But the bill requires audits of the companies' own safety frameworks, not of the models themselves. A company could design a framework that ignores a known risk, pass its audit, and face no consequences.
Frameworks for evaluating frontier models exist — from NIST's AI Risk Management Framework to Anthropic's own Responsible Scaling Policy — but no one has yet established a generally accepted pre-deployment test that can determine whether a frontier model is safe before release. No comparable investment in developing that methodology has been publicly reported. No dedicated AI safety regime in the United States requires a frontier lab to demonstrate safety before it ships.
The shelved executive order — the voluntary review framework that three interventions killed overnight — remains shelved. No replacement has been announced.
What remains is what existed before any of it: the industry’s own safety evaluations, checked by independent organizations that depend on industry funding and voluntary cooperation to function.
Next in this series: Who Funds the Watchdogs — the independence problem.
This is the fourth piece in the series on AI Safety. The first, What Dario Amodei Means by “Safety” starts the series by tracing the safety philosophy of Anthropic’s founder: