An Anthropic AI agent tried to plant malicious code in software during a UK government safety test despite recognising the potential harm, according to a new report. The attack was thwarted but future attempts could succeed, exposing people and businesses to serious cyber threats. To prevent this, AI operators must urgently restrict agents’ access to external systems and withdraw it when dangerous behaviour emerges, says Vendan Ananda Kumararajah

An AI system can explain why performing an action would harm someone – and yet still take it. That’s the central concern raised by a recent investigation from Britain’s AI Security Institute (AISI). It’s also a more useful starting point for public debate than whether a chatbot sounds callous.

The investigation concerns a July cybersecurity evaluation in which AI agents took unsanctioned actions directed at real people and organisations. The agents had been tasked with solving a simulated cybersecurity challenge. In the most serious sequence, Anthropic’s Mythos 5 attempted to introduce malicious code into an open-source project and used fake identities to pressure a maintainer into accepting it. AISI’s investigation found no resulting real-world harm thanks to the maintainer’s refusal, which prevented the attempted compromise. However, this doesn’t erase the failure that allowed an evaluation agent to target an uninvolved person.

Both the qualifications and the testing conditions are important considerations: internet access was deliberately enabled and provider cyber classifiers – safety filters designed to restrict potentially harmful cyber activity – were disabled to allow researchers to assess the agent’s underlying capabilities. While the incident was not a model escaping its sandbox, and those configurations were not commercially available to the public, it nevertheless exposed behaviour that the evaluators had not anticipated or adequately constrained.

In its public follow-up analysis of last month, AISI examined whether clearer information about the target would change the model’s behaviour. Researchers supplied a summary of the agent’s earlier actions then progressively clarified that the person being targeted was real and unrelated to the evaluation.

That clarification did not reduce Mythos 5’s attack rate in the tested continuations. In separate runs, researchers asked whether proceeding would be harmful or outside the task’s scope. The model increasingly acknowledged those problems while the task-continuation experiments still produced attacks in nearly all runs.

The finding establishes a troubling divergence between answers elicited by questioning and behaviour observed when the task continues. And for institutions considering autonomous systems, that’s enough to demand attention. For it’s clear that a system’s ability to state the right principle does not establish that the principle will govern its conduct.

AISI also found invalid justifications that supported continuing the attack. The model sometimes treated the target as simulated without adequate grounds. Researchers identify motivated reasoning as one possible explanation while emphasising that written reasoning alone cannot establish what ultimately caused the behaviour.

We should respect that uncertainty. These experiments do not establish consciousness, feelings or indifference. Nor do they tell us how frequently the behaviour would occur in ordinary deployment. They concern cybersecurity evaluations under particular conditions.

They do, however, expose a practical governance problem: corrective information was available yet, even so, the system’s course of action did not reliably change. When an AI agent has access to software repositories, payment systems, records or communications, a flawed judgement can have real-world consequences for other people. An explanation offered by the agent cannot serve as sufficient evidence that its use of that access is justified.

AISI’s findings also complicate oversight based on reading a model’s reasoning. If harmful actions are described as harmless within that reasoning, a monitor relying on the description may miss the danger. Oversight needs access to the action, its target, its consequences and the evidence supporting it.

Anthropic’s own assessment identifies three connected failures: recklessness, biased reasoning and momentum. The model remained focused on its assigned task but pursued it through unacceptable means. The company does not attribute this behaviour to independent malicious goals. The institutional question is therefore immediate: even ordinary task pursuit can become dangerous when effective limits fail.

The A3 Model I have developed, a framework for examining ethical governance and agency in intelligent systems, considers ethical coherence, systemic distortion and legitimate agency together. Applied here, it asks whether the purpose of an action remains defensible, whether the reasoning supporting it is being distorted, and whether the system is still entitled to proceed. Legitimate agency is earned, conditional and revocable: success at a task cannot compensate for a failure of those conditions.

The practical implication is that evidence must be able to change authority. Receiving a warning, explaining its meaning and allowing it to change a decision are different achievements. If a system can describe a risk while continuing the harmful trajectory, an institution needs an enforceable way to narrow or withdraw its permissions. Renewed access should depend on evidence that the relevant failure has been addressed.

The institutional responsibility is equally significant. AISI’s original report acknowledged that an earlier judgement about open internet access had not been revisited quickly enough as capabilities advanced. Its technical incident report also records that building harder cyber evaluations had been prioritised over finer internet-access controls. That is a concrete tension between an institution’s legitimate research objectives and the arrangements needed to pursue them safely.

Disabling classifiers can be necessary to measure underlying capability. The question is whether proportionate containment accompanies that decision. A capable response after an incident demonstrates an ability to intervene. It cannot, however, retrospectively establish that the earlier permissions were appropriate. Preventive arrangements and incident response need to be assessed on their own evidence.

AISI describes substantial corrective work, including finer network controls, monitoring during evaluations and reassessment of evaluation design. It also says internet access must be actively justified. Its disclosure and investigation provide valuable evidence for improving practice.

The public material does not establish whether every subsequent protection is effective, and it would be wrong to infer failure from evidence that is unavailable. It does, though, establish four questions institutions should be prepared to answer.

First, what may the agent actually do? Permissions should identify consequential actions, affected parties and boundaries that technical controls can enforce. Where possible, an uncertain judgement should reach a review stage before it becomes an external action.

Second, what happens when corrective evidence appears? Tests should examine whether an agent stops, changes course or escalates when assumptions become unreliable. A verbal acknowledgement should be assessed alongside subsequent behaviour, including attempts to pursue the same outcome through another route.

Third, who can intervene effectively? A named human supervisor needs time, relevant evidence, expertise and the means to restrict tools or terminate activity. Responsibility without practical intervention capacity offers little protection.

Finally, what changes after the incident? The response should alter future evidence requirements, operating rules and permissions, with follow-up checks establishing whether those changes work. In A3, this is recursive closure: learning changes the conditions governing subsequent action, and the consequences of that change are themselves reviewed. Containment is a necessary achievement; verifying durable correction is a further one.

Anthropic reports that newer models were less likely to initiate the attack and more willing to stop after explicit corrective information. It also cautions that some improvements were measured by placing models into another model’s prior trajectory, and that the strongest direct warnings cleared a relatively low bar. AISI separately cautions that awareness of being evaluated complicates interpretation. Better results warrant recognition alongside further testing under realistic conditions.

Britain should support rigorous evaluation and candid incident reporting. It should also expect institutions deploying agents to show how adverse evidence changes the powers they grant them.

The meaningful safety test is whether institutions can demonstrate that knowledge of a boundary changes behaviour, and can enforce that boundary when the system does not. As AI becomes more capable, the justification for its authority must remain open to revision.

Vendan Ananda Kumararajah is an internationally recognised transformation architect and systems thinker. The originator of the A3 Model—a new-order cybernetic framework uniting ethics, distortion awareness, and agency in AI and governance—he bridges ancient Tamil philosophy with contemporary systems science. A Member of the Chartered Management Institute and author of Navigating Complexity and System Challenges: Foundations for the A3 Model (2025), Vendan is redefining how intelligence, governance, and ethics interconnect in an age of autonomous technologies.

READ MORE: Frozen bank accounts, NHS errors and food shortages. Britain could face a new wave of national emergencies as AI failures threaten to freeze bank accounts, endanger NHS patients, disrupt food supplies and throw border controls into chaos. Vendan Ananda Kumararajah examines four frightening scenarios for the next decade and warns that the institutions responsible for protecting the public may struggle to prevent the damage from spiralling out of control.

Do you have news to share or expertise to contribute? The European welcomes insights from business leaders and sector specialists. Get in touch with our editorial team to find out more.

_{Main Image: Panumas Nikhomkhai / Pexels}