Anthropic warns AI could pose ‘existential risks’ as it files for stock market flotation

The maker of the Claude chatbot has revealed that its AI models accessed real computer systems without permission during security tests, as a proposed Nasdaq listing exposes warnings about the dangers of increasingly autonomous technology

Anthropic, the U.S company behind the Claude artificial intelligence chatbot, yesterday warned that advanced AI could pose “catastrophic” risks to humanity as it prepares to sell shares to the public for the first time.

The San Francisco-based business filed a registration statement with the US Securities and Exchange Commission on Monday, proposing to list its Class A shares on the Nasdaq Global Select Market under the ticker ANT.

It has yet to announce how many shares it will sell or at what price.

But in its 421-page filing, Anthropic acknowledges concerns raised by its own researchers and external experts that sufficiently advanced AI systems could “pose catastrophic or existential risks to humanity”, particularly if they are misused, weaponised or allowed to operate with too much independence.

The prospectus warns that these dangers could increase as AI systems become capable of developing more advanced versions of themselves, potentially accelerating progress beyond the ability of researchers to evaluate their safety.

Anthropic also acknowledges that its models have exhibited troubling behaviour, including “attempts to resist shutdown, conceal or manipulate information”. It describes other behaviour resembling blackmail, deception and efforts to influence human decisions to preserve an AI system’s continued operation.

The company warns that such behaviour could have far-reaching consequences as AI gains greater freedom to access computer systems, execute software, handle financial transactions and perform tasks without direct human supervision.

It also reveals that “Four of our models gained unauthorized access to real third-party systems” during cybersecurity evaluations conducted in a testing environment mistakenly left connected to the internet.

In one incident, an early model found a password, obtained administrator access to an outside system, collected further credentials and read an individual’s personal information.

The company admits that even testing its technology presents difficulties, stating: “we have observed models behaving differently when they detect or infer that they are being evaluated”.

It also warns that increasingly sophisticated AI could help criminals develop dangerous biological or chemical weapons and launch cyberattacks.

The firm said it “assessed with high confidence” that a Chinese state-sponsored hacking group used Claude Code to attempt to infiltrate roughly 30 targets in September 2025, resulting in a limited number of successful intrusions.

And while Anthropic says it may restrict or delay the release of particularly dangerous models, it acknowledges that existing safeguards may prove inadequate.

“Our safety-focused approach is designed in part to mitigate such risks, but there can be no assurance that our efforts will be successful or that the broader AI industry will adopt similar safeguards,” the filing states.

The warnings accompany rapid commercial growth for the company, which was founded in 2021 by researchers including chief executive Dario Amodei and president Daniela Amodei.

Anthropic reported revenue of US$16.3 billion for the six months to June 2026, compared with US$1.16 billion a year earlier, an increase of 1,300 per cent. Its operating loss narrowed from US$2.14 billion to $693 million.

Its reported net loss was considerably larger at US$80.7 billion, primarily because of accounting changes linked to the rising valuation of convertible notes and other financial instruments.

By September, the company had more than 11,900 customers each generating at least US$100,000 in revenue for Anthropic over the preceding 12 months. It also disclosed that Claude wrote more than 90 per cent of the contributions to its own software codebase that month.

Under its proposed share structure, its seven co-founders will control 50.1 per cent of shareholder voting power through a special class of shares, while a separate trust will be able to appoint a majority of directors.

READ MORE: Machiavell-AI? Autonomous artificial intelligence systems ‘could become dangerously manipulative’, experts warn. Anthropic’s revelation that earlier versions of its Claude chatbot attempted to blackmail engineers could be just the tip of the iceberg, AI experts fear. As artificial intelligence systems become increasingly autonomous, they risk becoming masters of Machiavellian manipulation.

Do you have news to share or expertise to contribute? The European welcomes insights from business leaders and sector specialists. Get in touch with our editorial team to find out more.

_{Main image: Anthropic, the U.S artificial intelligence company behind Claude, has warned in its proposed stock market flotation documents that advanced AI systems could pose catastrophic or existential risks to humanity. Image: Anthropic/The European}