Artificial intelligence is now central to strategic decision-making in sectors from healthcare to defence, but its outputs are only as reliable as the assumptions and oversight that guide them, warns digital transformation expert and acclaimed author, Marco Ryan. Effective leadership in this context calls for digital intelligence: the ability to ask informed questions, recognise when outputs don’t reflect reality, and stay alert to how automated decisions are shaped
Not so long ago, Amazon trialled a recruitment tool that downgraded CVs mentioning the word ‘women’. Elsewhere, residents in predominantly Black neighbourhoods of the United States were routinely charged more for car insurance because the systems involved relied on historic data, which mirrored existing social inequalities. Google’s image generator, Gemini, meanwhile, came under criticism when users discovered it would not produce pictures of white people, raising concerns over the way generative models interpret instructions and training inputs. These are examples of what happens when complex systems are deployed without enough thought about the consequences or the assumptions built into them.
Artificial intelligence is often introduced with the promise of speed, efficiency and consistency. In practice, however, it can amplify mistakes if organisations are not paying close attention to how these systems work, how they interpret data, and what kind of decisions they are being allowed to influence.
Technology is, after all, only ever as useful as the judgement that surrounds it, and that judgement now needs to be informed by how these systems are built, trained and applied. The environments in which AI operates are too dynamic, too opaque, and too consequential to rely purely on intuition or experience. Leaders must now cultivate digital intelligence to recognise how systems are trained, where risks and bias might arise, and how decisions are being shaped. That means being digitally curious, challenging assumptions, and being willing to invert traditional hierarchies so those closest to the technology can speak up.
In one hospital group, for instance, a predictive maintenance system was brought in to monitor critical medical equipment and anticipate faults before they happened. The engineers responsible for this machinery quickly noticed that the system was overreacting. Routine background readings were being flagged as signs of imminent failure, and devices that were working normally were being pulled out of service for unnecessary checks. This had the potential to cause more harm than it prevented, particularly if vital equipment was taken offline during periods of real demand.
Rather than follow the alerts without question, the engineers paused and reviewed the system’s recommendations. Their investigation revealed that the model had misinterpreted ordinary sensor fluctuations as warning signs. After retraining the system with human input, the frequency of false alerts fell sharply. The technology remained in place, but it now operated under closer scrutiny, with the benefit of practical understanding.
In a separate case, a senior British army officer was presented with drone footage of two figures crouched by a roadside known for insurgent attacks. The scene matched the pattern of previous threats, and the system had flagged the individuals as likely combatants. Under pressure to act quickly, the officer made the decision to wait. A few moments later, two children stood up and ran down the road. The images had been accurate, but the interpretation was wrong. It was not the algorithm that avoided disaster, but the individual who was prepared to question it.
Both situations involved people with relevant experience who recognised the limitations of the technology they were being asked to trust. Neither needed to understand the technical workings of machine learning or predictive models. What they brought to the situation was a clear understanding of context, a sense of consequence, and a willingness to take responsibility – all hallmarks of strong digital intelligence. Or, to put it another way, they weren’t following data but interrogating it.
Many organisations still treat AI tools as if they operate independently of human influence. The systems appear precise, and the data looks clean, but the outputs reflect a series of choices about which data to use, how to weight it, and what success looks like. These choices are often made without transparency, and without any clear process for reflection or correction when something seems off.
Younger employees tend to be more confident with digital systems and are often better equipped to understand how they behave. However, they rarely have the authority to override a recommendation or question an output. At the same time, senior leaders are asked to approve strategies and technologies that they may not fully understand. This division creates a gap between technical knowledge and decision-making power, a gap that only digital intelligence can bridge.
By ‘digital intelligence’, we mean more than a mere familiarity with digital tools. Rather, it’s a leadership quality grounded in digital curiosity, ethical focus, and a willingness to keep learning. It doesn’t require coding knowledge, but it does demand sharp questions, an ability to spot unexpected outcomes, and the judgement to know when something isn’t right. Digital intelligence also means fostering the kind of environment where people can raise concerns, disagree openly, and remain accountable for the systems they oversee.
With this in mind, strong leadership in a digital environment isn’t defined by how quickly new tools are adopted, but by how well they’re understood and governed. It means embedding processes for scrutiny and reflection, ensuring those with insight are heard, and stepping in when things go wrong. Crucially, it includes knowing when to pause or step back rather than push ahead.
As I argue in Rewire or Retire, my new book, the pressure to embrace new technologies must be balanced with a capacity for ethical reflection and situational judgement. Retire doesn’t mean giving up but knowing when the costs outweigh the benefits, when assumptions need to be challenged, and when speed must give way to care. Sometimes, the responsible choice is to pause, step back, or even stop. In an age of accelerating complexity, the Retire option matters just as much.
AI will continue to play a central role in business, government and public life; its influence will grow, and its capabilities will improve. But the need for human judgement will not disappear. Leaders remain responsible for what happens under their watch, whether or not a machine was involved. The real test of leadership is not how quickly someone adopts a new tool, but how well they understand the systems they rely on, and how prepared they are to challenge them when it matters.
Marco Ryan is an experienced Non-Executive Director, digital advisor and former Chief Digital Officer at BP. He is Cyber Leader in Residence at Lancaster University Management School and teaches on its Cyber Executive MBA. His latest book, Rewire or Retire, helps board members and senior leaders understand the impact of AI on leadership, offering clear strategies for adapting in an age of rapid digital change.
Main image: Courtesy Ron Lach/Pexels
