A major MIT study has mapped how serious the risks from artificial intelligence could become. But identifying the dangers is only part of the problem. Vendan Ananda Kumararajah argues that as AI systems gain greater autonomy, governance must also determine whether the systems – and the institutions controlling them – possess legitimate authority to act

Artificial intelligence does not suffer from a shortage of identified risks. A recent MIT FutureTech Delphi study, drawing on 272 international experts, found that 18 of 24 AI risk domains were judged to carry at least a 10 per cent probability of catastrophic outcomes under business-as-usual assumptions over the next five years. 

Even under pragmatic mitigation scenarios, dangerous capabilities, weapons and cyberattacks, environmental harm, inequality and unemployment and power centralisation remained above that threshold.

The study also exposed a deeper asymmetry: those most vulnerable to AI harms are often ordinary users and affected stakeholders, while responsibility for mitigation lies primarily with powerful developers and governance actors.

AI governance has become increasingly sophisticated at identifying what can go wrong. A different question now becomes unavoidable: why do these risks keep clustering and who has the legitimate authority to act when AI systems become increasingly autonomous?

That is the question I examine in my recent SSRN paper, From AI Risk Taxonomies to Recursive Cybernetics: The A3 Model as a Category and Order Shift in Systemic AI Governance. My argument is that risk taxonomies give us the surface of the problem but not necessarily its underlying governance grammar.

Disinformation, dangerous capability, power concentration, unsafe deployment and governance failure may appear to be separate risks. Yet beneath them can lie recurring conditions: distorted incentives, weak epistemic integrity, ethical incoherence and capability outrunning legitimate agency. That requires a shift from governing risk alone to governing agency itself.

As AI systems become capable of coding, diagnosing, recommending, allocating resources, persuading and acting through tools, governance naturally focuses on whether they can perform those tasks safely.  A3, however, asks an earlier question: should the system possess that degree of agency at all?

This is the role of one of the A3 Model’s three core concepts, Adhikaram, or legitimate agency. Adhikaram distinguishes technical capability from the right and fitness to act. It asks whether the system, institution or human actor has sufficient maturity across Knowledge, Action, Experience, Absorption and Governance to exercise consequential authority.

An AI system may therefore be extraordinarily capable while still lacking legitimate agency. So may the organisation deploying it, and so may the regulator overseeing it. Formal authority is not the same thing as governance fitness.

The MIT study also points towards competitive dynamics, power centralisation and governance failure. These risk categories reveal something more systemic.

A company may know that an AI system has unresolved weaknesses while simultaneously facing investor pressure, competitor releases and first-mover incentives. Every actor can behave rationally according to local incentives while the system as a whole moves towards an irrational outcome.

A3 describes this condition through Aanavam, or systemic distortion. It includes competitive pressure, institutional self-interest, opacity, capture, incentive corruption, false certainty and the normalisation of harmful behaviour. If the feedback system itself is distorted, simply adding more monitoring may not solve the problem because the feedback can reproduce the distortion.

The third A3 primitive is Aram, or ethical coherence.

AI ethics has already produced important principles around fairness, accountability, transparency, privacy, safety and human autonomy. But principles alone are insufficient if they remain external to the operating logic of the system.

A system can be technically successful and economically viable while transferring unacceptable costs to workers, citizens or future generations. It can adapt while preserving injustice, learn from distorted experience and survive by externalising harm, so viability alone is not enough.

A3 places Aram, Aanavam and Adhikaram together because ethical coherence, distortion awareness and legitimate agency must remain synchronised.

Safety, compliance and evaluation remain essential, but AI governance must also examine whether a system’s purpose is legitimate, what distortions are shaping its behaviour, whether its knowledge is adequate to the consequences involved, whether capability has exceeded governance maturity and whether those governing the system are themselves fit to govern. It must also determine what happens when legitimacy deteriorates.

A governance architecture for increasingly autonomous AI must be capable of more than observing failure. It must also be capable of changing the system’s authority to act through correction, containment, reduced autonomy, escalation, suspension or reconstitution.

The A3 architecture developed in my SSRN paper translates these ideas into instruments including the Knowledge Continuum, ethical viability testing, distortion tracking, agency fitness assessment and governance posture states.

Perhaps the most difficult implication is that recursive governance cannot stop at the AI system. Developers, boards, regulators and governments must also remain subject to the legitimacy test.

A regulator may have statutory authority while lacking sufficient knowledge or independence to govern frontier systems effectively. A developer may possess enormous technical competence while operating inside incentives that weaken restraint. A board may formally approve a deployment whose consequences it does not adequately understand. The legitimacy question therefore travels upwards to who governs the governor when the governor itself becomes part of the distortion.

The MIT study demonstrates how serious and interconnected the AI risk landscape has become. The next step is to develop a governance architecture capable of explaining why these risks reinforce one another and of determining when consequential agency remains legitimate, moving from risk enumeration to recursive governance.

As AI becomes increasingly autonomous, one principle may become fundamental: capability must never be allowed to substitute for legitimacy.

Vendan Ananda Kumararajah is an internationally recognised transformation architect and systems thinker. The originator of the A3 Model—a new-order cybernetic framework uniting ethics, distortion awareness, and agency in AI and governance—he bridges ancient Tamil philosophy with contemporary systems science. A Member of the Chartered Management Institute and author of Navigating Complexity and System Challenges: Foundations for the A3 Model (2025), Vendan is redefining how intelligence, governance, and ethics interconnect in an age of autonomous technologies.

READ MORE: ‘The AI disaster nobody sees coming‘. Europe’s AI rulebook is taking shape, but what if the next major failure comes not from a lack of compliance but from governance systems that appear sound while drifting out of control? Vendan Ananda Kumararajah, creator of the A3 Governance Dashboard framework, argues that compliance alone may not be enough to detect governance drift.

Do you have news to share or expertise to contribute? The European welcomes insights from business leaders and sector specialists. Get in touch with our editorial team to find out more.

_{Main Image: Pexels/Pixabay}