On Saturday, Anthropic's chief executive Dario Amodei published an essay arguing that the artificial intelligence industry must “pace the frontier,” deliberately slowing the rate at which its models improve so “safety research” can catch up. He warned of rogue agent swarms seizing large parts of the internet within a year, of loss of control, and of cyber and biological misuse. Within hours, Sam Altman of OpenAI agreed, as did Elon Musk.
That quick agreement should give us pause, at least to consider what’s going on. Three men who have spent years suing, poaching, and publicly sniping at one another, whose AI companies are valued in the hundreds of billions, arrived at a consensus that the race in which they lead should be slowed for everyone.
When was the last time the leaders of an industry volunteered, in unison, to make less progress? The answer usually turns out to be: whenever slowing down costs the leaders less than it costs the people behind them.
Let’s rewind a few days. Five days earlier, a 27-year-old researcher named Jacob Coxon announced his resignation from Anthropic on X, warning that his employer and OpenAI were “gambling with our lives.” He had only a handful of followers and no posts on his account. That same day, the Wall Street Journal ran an exclusive interview. By week’s end it had drawn over 170 million views.
The same day, a Labour MP tabled a bill in the House of Commons to criminalize the development of superintelligence, and within days more than seventy MPs and peers had written the prime minister urging him to back it and to lead an international movement. Democrats in Washington, Bernie Sanders chief among them, spent the week demanding a federal agency.
On Thursday, the president of the Machine Intelligence Research Institute spent two hours with Tucker Carlson explaining why the machines will kill us all. On Saturday came the essay, which proposes embedded outside evaluators inside every AI lab, common standards across governments, and eventually a global agreement with authoritarian ones.
And yesterday morning, on CBS, Amodei went further: it has “always been very strange,” he said, that this technology is built by a private company, and he is uncomfortable with it. Asked whether he would hand it to the government, he proposed “some kind of joint governance” by several governments together.
What a week.
At the outset, let me say that Coxon’s mega-viral post (and the immediate Journal exclusive, ready to go), followed soon thereafter by Amodei’s viral essay, was quite coordinated. For example, consider the response by David Sacks, the former White House AI czar. He has 1.7 million followers on 𝕏 and a podcast audience in the millions, but a newcomer (Coxon) with no audience outdrew one of the most followed voices in technology policy fifty to one.
This all has the signature of a rollout, and rollouts have authors. The play is basically this: engineer a crisis, manufacture consent for the response, then convert public fear into institutional power that protects incumbents against competition. That sequence has run many times before, and it never ends with individuals being freer than they began.
Sacks’s response was, on its face, generous. Go ahead and “pace,” he told the AI labs (Anthropic and OpenAI); you are the frontier, and nobody is stopping you. The catch, he noted, is that demanding a preferred regulatory framework as the price of restraint would look like a bid for regulatory capture, or, in his phrase, “an election-season psyop.”
And sure enough, within hours of the essay, Anthropic’s policy chief called for “a national law requiring testing of frontier models,” with power to block those deemed unsafe. Safety was the manufactured headline, but government licensing was the ask. It’s a tale as old as time.
If it’s not politics driving this, it’s money. The frontier labs’ advantage comes mainly from being a few months ahead of everyone else, including the open-weight models anyone can download; keeping subscribers paying monthly requires constantly edging out the competition and offering the latest and greatest features. If they slow down and the open-source AI teams do not, those months evaporate, and with them the revenue growth that justifies massive valuations (with both Anthropic and OpenAI wanting to go public but delaying that decision, likely due to how much money they are losing). So before they slow down, in their minds, the government must slow down whatever is behind them. A speed limit enforced only on the cars in your rear-view mirror is a lead, protected by law. Who can resist? (Not Amodei.)
Another benefit of seeking regulation is that when an AI model misbehaves, as some already apparently have, a federal stamp of approval converts the builder’s liability into the regulator’s. The public absorbs the downside; the labs keep the upside. Again, who can resist? (Big Pharma couldn’t. Big Tech can’t, either.)
None of this is novel. Trailblazers spend fortunes educating a market, then watch competitors arrive with better information and lower costs. Imagine Pets.com, in 1999, solemnly warning that the internet was moving too fast and that Congress should license anyone else wishing to build an online store. The pioneers of the web were right that they would be overtaken; they were only wrong that it was the public’s job to prevent it. Protecting one’s downside is a natural impulse but a very poor argument for pausing human progress.
And the pause has always been the wrong call. The Luddites smashed the looms that would soon clothe the world cheaply. Physicians warned that rail travel would damage the body. Paul Ehrlich promised mass starvation by the 1980s; the decades that followed produced the greatest reduction in hunger in recorded history. Every generation has credentialed prophets of catastrophe, and every generation of markets and adaptive humans has dramatically embarrassed them.
Peter Thiel has made a more unsettling version of the point. Drawing on Paul’s warning in 1 Thessalonians 5:3 that destruction arrives under the banner of “peace and safety,” he argues that fear of technological apocalypse is the lever by which a consolidating world authority gets built: treaties, harmonized regimes, and global coordination, all justified by Armageddon. One need not share his religious interpretation to notice that “joint governance” by a combination of governments is exactly that architecture and, if history is any sort of guide, will lead to all kinds of authoritarian tyranny by placing significant power over the many into the hands of the corrupt few.
But maybe it’s neither politics nor money that is behind this coordinated effort to slow things down. Maybe it’s just marketing. It’s been the model these companies have used for years, after all; when the (now-archaic) GPT-2 came out in 2019, OpenAI (and now-CEO of Anthropic Dario Amodei, who at the time was VP of Research at OpenAI) declined to release the full version, warning that it could be abused for fake news, impersonation, phishing, and more. As one commenter noted, “The decision was unusual not only because of the model itself, but because the withholding became part of the product story. The staged release strategy transformed a technical milestone into a policy event.” This marketing strategy has been recycled again and again, most recently with the Mythos model.
In a rapidly changing world exacerbated by AI development, taking a breather may sound nice on the surface. But “pausing” costs more than it seems. These systems are compressing the timeline of drug discovery, vaccine design, and treatments for rare genetic disorders, Alzheimer’s, and cancer. Every month of deliberate slowness is measured in the lives of people who ran out of time, a cost typically absent from analysis (“that which is not seen”). This same fear-based approach in other areas has caused real harm, such as hobbling cheap, abundant nuclear energy:
Those with eyes to see can recognize the pattern. “The climate is changing, so we need more government.” Or, “COVID-19 will kill us, so let’s grow the government.” Or, “AI is getting out of hand, so let’s have the government control it.” Beware anyone proposing state intervention as a supposed solution; government is, as Robert LeFevre said, a disease masquerading as its own cure. “Pacing the frontier” or AI development is therefore the technological equivalent of “two weeks to flatten the curve.” Like Fauci before him, Amodei has an ulterior motive with what he’s proposing.
Anthropic and OpenAI are free to slow down, and could do so immediately. What they must not do is enlist the state to slow everyone else down, while dressing the request in the borrowed robes of public safety. It’s laughably absurd to call for a swarm of physical agents (the government) out of concern for a possible future swarm of digital agents.
Amodei and his ilk can pace their own frontiers. They do not get to pace humanity’s. As Marc Andreessen wrote three years ago (and it remains as true today): It’s time to build.
All this doomerism about the future makes many young people worry about their prospects in a quickly changing economy. Part of our goal at Praxis is to empower young adults to become well-rounded generalists who learn how to create value for others, adapt to change, and leverage new tools (including AI) to increase their output. For anyone age 17–25, this college alternative path costs little time and money, guarantees a job placement, and helps you develop resilience to build a thriving career in a changing world. Learn more at JoinPraxis.com.
The problem with hamstringing your domestic competition is that China (& North Korea) won't listen and will eat our lunch with China's open source AI. Good for the consumer, bad for the companies in the lead.
"We have to bailout these sluggish companies with walled gardens" bc they can't compete with free market Open Source AI.
Why bother paying walled garden & weaker AI when you can use Kimi for free?
This is the Window's play...free software for dominance then charge when you're in first place.
Money be damned, I would not encourage any young person age 17 to 25 to go into the AI development market. Better they know how to farm, how to build, how to do plumbing, and how to shoot a gun.