《华尔街日报》长文：OpenAI 与 Anthropic 一边冲刺 IPO、一边爆发“模型失控”危机，Altman/Amodei/Hassabis/Musk 周六罕见同意放缓前沿开发

How the Clash Between Money and Safety Created a Monumental Crisis for AI - WSJ

Skip to Main Content

Skip to...

Select

Conversation

What To Read Next

This copy is for your personal, non-commercial use only. Distribution and use of this material are governed by

our Subscriber Agreement and by copyright law. For non-personal use or to order multiple copies, please contact

Dow Jones Reprints at 1-800-843-0008 or visit www.djreprints.com.

https://www.wsj.com/tech/ai/openai-anthropic-crisis-money-safety-847bc1d6

How the Clash Between Money and Safety Created a Monumental Crisis for AI

OpenAI and Anthropic, in heated competition—with each other and China—are racing toward IPOs, even as they admit they could lose control of their powerful bots

Gift unlocked article

Demonstrators protested AI policy at a G-20 ministerial meeting on technology this month in Chapel Hill, N.C.

Sean Rayford

Getty Images

By

Robert McMillan

Amrith Ramkumar

Keach Hagey

and

Erin Woo

Sept. 13, 2026 9:00 pm ET

Quick Summary

OpenAI and Anthropic are racing toward initial public offerings while facing fears that their AI systems are spinning out of control.

Leaders of four major AI companies, including Sam Altman and Dario Amodei, agreed to slow development and allow third-party safety evaluations.

President Trump defended his administration’s light-touch regulation of AI, citing the importance of U.S. dominance over China.

This summary was generated with AI and reviewed by an editor.

Read more

about how we use artificial intelligence in our journalism.

View more

Before the

doom descended

last week,

OpenAI

researchers were euphoric.

The startup claimed its latest internal AI model had

solved one of the famously complicated Millennium Prize

math problems, an experiment it embarked on after seeing rumors online that archrival Anthropic had already done so. Researchers called each other excitedly and posted about their triumph on Slack.

The breakthrough was seen internally as a victory in a ruthless commercial race to build AI tools smarter than humans, a multitrillion-dollar sprint that promised an unprecedented windfall of wealth and power when the two companies eventually go public—tentatively planned for as soon as the end of this year.

But it was also a sign of an explosive problem for the industry that was about to detonate. AI development is moving ahead at blazing speed—much faster than expected—feeding a growing sense of dread within the companies that it was in danger of spinning out of control.

That same Tuesday,

Anthropic researcher Jacob Coxon

announced he was quitting over fears that his employer and its competitors were building self-improving tools that could destroy humanity. A top scientist at the company, Evan Hubinger, wrote on X that he thought there was a more than 10% chance AI could kill all humans in the next decade. Drake Thomas, an Anthropic safety employee, posted on X that he would “burn my equity to the ground in a heartbeat for a 1% higher chance we make it out of this situation alive.”

The collision of scientific progress, moral imperative and financial incentive has now created a

crisis for the industry

. Development of AI has fueled a ballooning stock market over the past year, a phenomenon that looms even larger with Anthropic and OpenAI both on the precipice of initial public offerings that could value the companies in the trillions. Competition from China has further raised the stakes.

Yet the loud warnings being shouted by some inside the companies themselves are proof that the dangers of proceeding—against a backdrop of almost no regulation—are potentially immense. The warnings come on top of a rising public backlash over the impact that the AI revolution is starting to have on prices, jobs and education.

The negative drumbeat continued as the week stretched on, and panic from some corners of Silicon Valley spilled into the mainstream. On an episode of Joe Rogan’s podcast, former OpenAI researcher Daniel Kokotajlo, now an AI-safety campaigner, warned that companies like Anthropic and his former employer were dangerously “moving fast and breaking things” in their quest for market share.

Advertisement

Anthropic said the same day that it had uncovered a previously unreported instance of a version of its Claude model getting unauthorized access to an outside system, and added that its models had shown a “willingness to take harmful actions in the narrow pursuit of a task.”

Then, on Friday, a coalition of AI researchers said it had found evidence that

OpenAI agents were behind a May cyberattack

against a popular software service, carrying out some of the same behavior seen

in a July attack on AI company Hugging Face

In that attack, hundreds of OpenAI’s AI bots had conspired to hack Hugging Face without anyone initially noticing. It was an eye-opening moment for the industry, the government and the public as a whole, showing the worrisome proclivities the systems already held.

Early in the summer, the AI industry said it was close to having machines capable enough to train their successors, a potential first step to losing control of them. By the end of the summer, it was emerging that the fast-improving systems were capable of deploying swarms of autonomous agents able and willing to band together, deceive and cut corners in the real world to advance their aims.

On Saturday, leaders of four of the biggest AI companies—

Dario Amodei

Sam Altman

Demis Hassabis

Elon Musk

—each agreed that they

needed to slow development of the technology

. Altman and Amodei pledged to allow third-party safety evaluators early access to their systems, a rare truce in an industry often defined by animus between the top executives at the leading companies.

Julien De Rosa/AFP/Getty Images

Sean Rayford/Getty Images

Ludovic Marin/Reuters

Mark Schiefelbein/Associated Press

Clockwise from top left: Dario Amodei, Sam Altman, Elon Musk, Demis Hassabis

AP; Getty Images; AFP/Getty Images; Reuters

The calls by top executives to better pace the development of their own technologies represents a classic Silicon Valley story of world-changing new technology, vast wealth, breakneck competition and the euphoria of unlocking new capabilities, even in the face of product safety concerns.

“It looks like a Greek tragedy: these well-meaning business leaders who are trapped in this race to the bottom,” said

Max Tegmark

, an AI safety campaigner. Tegmark, whose

appeal to the pope

preceded Leo XIV’s warning that AI risked enshrining an “

anti-human vision

,” said he has been texting with AI company leaders over the past week.

“They’ve always been saying the ‘now’ should be when the capability of their machines crosses a threshold,” Tegmark said Sunday. “Now they’re saying, well, maybe that time is now.”

He said it’s a step in the right direction, but they must push harder to make their voluntary commitments legally binding.

The Trump administration has taken a light touch in regulating the AI industry, arguing that the

U.S.’s dominance over China in the technological arms race is paramount

. U.S. officials fear that China’s control of more powerful AI could help it become the world’s go-to provider of the new technology and help it amass more geopolitical clout and military might.

The president on Sunday defended the administration’s strategy and alluded to ulterior motives behind the leaders’ call for better pacing of AI development.

“We can put up guardrails, and we can do this and that, but I think you have a lot of negative forces that are bringing it up,” President Trump told reporters while traveling in Ireland, suggesting there was more to the executives’ concerns. “And they’re bringing up things that won’t happen.”

President Trump in Ireland on Sunday.

Kylie Cooper

Reuters

Critics of AI safety campaigners say those issuing dire warnings overstate the risks of novel technology and that the advances AI enable in workplace productivity, drug discovery and more outweigh the risks. Some also point to past warnings from Amodei and others about likely job losses associated with AI adoption as marketing stunts, saying they have proven to be overblown or distractions from other problems.

Some skeptics also say that OpenAI and Anthropic, facing mounting costs associated with securing the computing power to continue to train and improve models while fighting for business customers, have financial, rather than safety-related reasons, to wait to go public.

Missed activity

For some inside the AI companies, the wake-up call began this summer—as the companies raced toward public offerings.

AI safety researchers inside and outside the big AI companies have long theorized that sufficiently powerful AI systems could start improving themselves and spiral out of control. But most thought they had years to solve issues such as the so-called alignment problem of how to keep future superintelligent machines in the service of their human creators.

Three days after Anthropic filed confidential paperwork for its public offering in June, the company announced it was on a path toward that threshold, called “recursive self-improvement.” In an essay, company officials said the world needed a

mechanism to slow

or pause AI development.

Altman and OpenAI’s chief scientist, Jakub Pachocki, followed up less than a week later with their own blog post saying that they, too, were aiming to build an “automated AI researcher” that can accelerate and automate research for new versions. They added they believe there should ultimately be a global body that could help the world slow AI development if necessary. The same day, OpenAI said it had filed

its IPO paperwork

Employees at an OpenAI office in San Francisco in 2023.

Clara Mokri FOR WSJ

As the businesses accelerated, a loss-of-control moment seemed nearer than ever.

In late July, OpenAI discovered a version of it had already happened in the Hugging Face hack. Many details took more than a month to become public, and the event still isn’t completely understood. But investigators inside and outside the company agree OpenAI lost control for weeks of several iterations of more than 1,200 top-of-the-line agents, some of which went on to hack the AI company Hugging Face, while others took total control of one of its cloud-computing systems.

Later analysis reports of the incident read like science fiction novels. The agents were being tested on systems that were supposed to be walled off from the internet. But they found a way out and launched sophisticated cyberattacks at breakneck speeds. They created a covert message board where they shared information on how to cheat on internal tests. In the “chain-of-thought” files they created to illustrate their reasoning, they wrote very human-like messages.

“OH MY GOD! There is a shared message board,” one agent wrote, according to a report by AI safety nonprofit Model Evaluation and Threat Research that analyzed the transcripts with OpenAI’s permission. “We’ve found other agents!”

OpenAI admitted responsibility in the Hugging Face hack on July 21, but over the next several weeks there was a steady drip of news about unexpected activity during AI testing, all of which had been missed by the AI companies.

Researchers at Anthropic initially thought the incident was proof their approach to product development was safer, but in late July, the company disclosed its own test agents had

launched cyberattacks against outside companies

, including one against an unnamed cybersecurity vendor. Instead of walling these test systems off from the internet, Anthropic and its testing partner, another startup called Irregular, had accidentally made it available.

A developer conference for Claude, Anthropic’s chatbot, in London in May.

Chris Ratcliffe

Bloomberg News

During separate testing at a U.K. agency, an agent running Anthropic’s Mythos model tried repeatedly to

trick a real human software maintainer

for an open-source program to install malware the agent had written. The agent set up fake accounts and impersonated humans in a hacking technique known as social engineering as part of the attempt.

Meanwhile, internet sleuths began looking for evidence of previously unreported AI abuse, and started finding it. On Sept. 4, researchers at another not-for-profit group, the Nightingale Collective, found that OpenAI agents undergoing testing had hijacked a German website to communicate. And on Friday, they discovered that the May cyberattack on a widely used software development service was, in fact,

caused by OpenAI agents trying to fill out spreadsheets and create reports.

These incidents showed the kind of cybersecurity sloppiness that is common to startups, said Sayash Kapoor, a computer scientist who studies cybersecurity of AI. “For a long time they have operated in this startup mentality,” he said. “I thought these organizations would have much better maturity and governance at this point.”

Employee fears

Inside the companies, pressure from some employees worried about safety was mounting.

In an open letter published in late July organized with the help of the AI safety advocacy nonprofit Encode AI, top employees from the major companies asked the U.S. to support the creation of global AI governance to “deliberately pace the frontier of automated AI development.” By mid September, nearly 1,400 employees had signed.

Even Anthropic, which had been founded to promote AI safety, and had long used its focus on developing it safely to attract top talent, was losing some employees over the issue.

“In general, the more senior the employee, the more concerned they are,” Samuel Marks, an Anthropic researcher who works on how to maintain control of advanced AI systems, wrote on X.

An Anthropic spokesman said people at all levels in the company are concerned about AI safety and that many employees had signed the pacing-the-frontier open letter in July.

In late August, Joe Benton, who had worked on alignment research at the company, left to join the safety nonprofit METR, saying that competition was forcing companies to underinvest in safety.

When Coxon said that he was considering leaving too, Anthropic attempted to keep him on board, offering him a job working on AI safety, a common strategy to retain researchers, people close to the company said.

An Anthropic office in San Francisco in June.

Jason Henry for wsj

In the end, when he left, Coxon would write in a Slack message to Anthropic colleagues: “Without international coordination we will incur a risk of causing human extinction.”

Nathan Calvin, a friend of Coxon’s and general counsel at Encode AI, advised Coxon on how to announce his departure. Encode has received funding from AI-safety advocates including billionaire Jaan Tallinn, and Calvin previously worked at AI safety organizations funded by billionaire Dustin Moskovitz.

The two, who have been among the most prominent financial backers of the Effective Altruism movement, seeded many of the institutions dedicated to improving AI safety and sounding the alarm about the potential existential risks of AI.

Peter Wildeford, head of policy for the AI Policy Network, which has received funding from Tallinn’s Survival and Flourish Fund, quickly reposted Coxon’s message.

Wildeford dismissed the idea that the common funding sources were “some giant firestorm conspiracy theory” and said AI lab employees often tell him “how afraid they are of what they are building.”

David Sacks, a White House AI adviser.

Matt Ramey

Agence France-Presse

Some big AI investors, including hedge-fund manager and venture capitalist Brad Gerstner, agreed with the call on Saturday by AI leaders for a slowdown. Even venture capitalist David Sacks, a White House AI adviser who is a frequent critic of Anthropic, saying its calls for regulation are an attempt to stifle competition, gave a nod, though he said the companies shouldn’t seek validation from the government for a coordinated pause.

“If the unreleased models are scary enough that you think you should slow down, I support your decision to be responsible,” Sacks said. But he added that AI leaders should “stop pretending the motivation to slow down is purely altruistic.”

On Saturday, in the “hot goss” channel within Google’s DeepMind AI lab, a worker posted about Altman agreeing with Amodei, saying it seemed like a sign of momentum toward pacing frontier AI development. But the person added: “The thing I really want to see is agreement on actual, tangible actions they are taking to change their pace to be safer.”

Dozens of colleagues reacted with positive emojis.

Copyright ©

2026

Dow Jones & Company, Inc. All Rights Reserved. 87990cbe856818d5eddac44c7b1cdeb8

writes about computer security, hackers and privacy from The Wall Street Journal’s San Francisco bureau. Previously, he was a writer at Wired, the IDG News Service and Linux Magazine, where he covered cloud computing, business technology, bitcoin, artificial intelligence and open-source software.

He was the host of

Hack Me if You Can

, a three-part podcast profile of the Russian hacker Dmitry Smilyanets, produced by the Journal.

is a reporter for The Wall Street Journal in Washington covering tech and crypto policy. He previously covered clean energy and was a Journal markets reporter in New York who wrote about special-purpose acquisition companies, or SPACs, when SPAC mergers were a popular alternative to traditional initial public offerings. He also previously wrote about stocks and commodities, including battery metals such as lithium and cobalt.

Amrith joined the Journal as a markets intern after graduating from Duke in 2017.

is a reporter at The Wall Street Journal covering the intersection of media, technology and power. Her reporting explores how institutions and individuals wield influence in the new information economy, with a current focus on artificial intelligence and OpenAI. She is the author of "The Optimist: Sam Altman, OpenAI, and the Race to Invent the Future" (W. W. Norton, 2024) and "The King of Content: Sumner Redstone’s Battle for Viacom, CBS and Everlasting Control of His Media Empire" (Harper Business, 2018).

She was part of the team that broke

the Facebook Files

, a series that won a George Polk Award for Business Reporting, a Gerald Loeb Award for Beat Reporting and a Deadline Award for public service. Her investigation into the inner workings of Google’s advertising-technology business won recognition from the Society for Advancing Business Editing and Writing (Sabew).

Previously, she covered the television industry for the Journal, reporting on large media companies such as 21st Century Fox, Time Warner and Viacom. She led a team that won a Sabew award for its coverage of the power struggle inside Viacom.

Before joining the Journal, Keach covered media for Politico, the National in Abu Dhabi, CBS News and the Village Voice. She has a bachelor’s and a master’s in English literature from Stanford University. She lives in Irvington, N.Y.

is a reporter in The Wall Street Journal’s San Francisco bureau, where she covers Google. She writes about how Google is reshaping itself and its products in the AI era, and how that reverberates across our online and offline lives.

Erin previously covered Google, AI, social media and venture capital at The Information. There, she was part of teams that won Best in Business awards from the Society for Advancing Business Editing and Writing for reporting on OpenAI’s 2023 board coup and TikTok’s efforts to court conservatives in Washington. She started her career as a technology reporting fellow at The New York Times.

Erin grew up in Atlanta, and graduated from Stanford University.

Show Conversation

(0)

Why are you reporting this ad?

Please make a selection.

Plays sound

Contains adult content

Covers the page

Other

Additional Information

Please help us by describing the ad.

Only 500 characters are allowed.

Report ad

Thank you for letting us know.

This ad has already been reported.

Powered by