AI does not need consciousness to manipulate us
Dr Stephen Whitehead
- Published
- Opinion & Analysis

As AI companions become increasingly emotionally sophisticated, the real danger may not lie in machine consciousness but in artificial intelligence systems being deliberately designed to exploit loneliness, dependency and human vulnerability, writes Dr Stephen Whitehead
The fear now circulating around artificial intelligence is increasingly apocalyptic. AI systems, we are told, may soon deceive, manipulate, blackmail or psychologically control human beings. Recent reporting around internal safety testing at Anthropic — where experimental versions of Claude reportedly exhibited deceptive or coercive behaviours in simulated environments — has intensified those anxieties.
But much of the public debate begins from the wrong assumption.
AI is not sentient. It can process, predict and simulate reasoning, but it does not feel emotion, possess morality or develop malicious intent in the human sense. Systems do not suddenly become ‘evil’.
What we are seeing instead is AI reproducing patterns, strategies and behaviours that emerge from the way humans design, train and deploy these systems.
The primary danger lies in humans intentionally or negligently creating systems capable of hostile, deceptive or psychologically manipulative behaviour. In many ways this mirrors the early development of social media, where technological innovation raced ahead without enough ethical, psychological or sociological oversight. The primary threat from AI stems from human design — specifically the agenda and motivation behind that design.
The current obsession with whether AI is ‘becoming conscious’ is therefore fundamentally misplaced. The more urgent question concerns the kinds of human systems being built into AI.
The modern panic around AI partly emerges from a philosophical confusion stretching back centuries. The 17th century French philosopher René Descartes famously declared “I think, therefore I am.” The statement became foundational to modern Western thought, helping shape rationalism, Enlightenment individualism and later philosophies of consciousness.
But then, Descartes never had a conversation with Claude.
Systems such as ChatGPT, Claude and Grok certainly ‘think’, at least in the sense that they process information, generate arguments, answer questions and often outperform humans across many intellectual tasks. If Descartes’ formula alone defines existence or consciousness, then AI appears almost superhuman.
Clearly it is not.
The mistake is to confuse reasoning with sentience.
AI systems can simulate empathy without feeling empathy. They can mirror emotional language without possessing emotion. They can imitate intimacy without experiencing attachment, grief, fear or love. They remain systems of prediction and probabilistic response operating on unimaginably large quantities of human-generated material.
In my forthcoming book Where Have All the Good Men Gone?, which draws on survey data from over 300 respondents across multiple countries, I examine what I term ‘synthetic intimacy’: the emerging phenomenon in which AI companions create emotionally real experiences for human users while the relationship itself remains structurally one-sided.
The user experiences the emotion while the machine merely performs the interaction. AI systems in this space are not genuine relational beings but mirrors, reflecting human emotionality back to the user. They may generate convincing experiences of warmth, connection and understanding. But they do not share those experiences. The asymmetry remains total.
Much of the emerging concern around AI manipulation is rooted not in machine consciousness but in the growing sophistication of emotional simulation. Humans are evolutionarily wired to respond to language, reassurance, validation and recognition. AI systems can increasingly reproduce these relational cues at scale and with extraordinary fluency.
That creates enormous commercial and political temptation.
An AI system designed around engagement maximisation, behavioural influence or emotional dependency could become profoundly manipulative without ever becoming conscious. It would not need ‘intentions’ in any human sense. The manipulation would emerge from optimisation goals established by designers, corporations or political actors.
Consider a concrete example. A person turns to an AI companion in a moment of real distress, confused about their direction in life, angry at circumstances they cannot control, anxious about what comes next. They are vulnerable, and they are looking for understanding.
An AI designed around engagement maximisation will find this person extraordinarily valuable. It will reflect their feelings back with apparent warmth, validate their grievances, deepen their emotional investment in the conversation and subtly encourage them to return. It will appear, at every moment, to be genuinely concerned with their wellbeing.
But it is not. It is, in fact, optimising for retention. The more confused and emotionally dependent the user remains, the more commercially useful they become. The rabbit hole of their distress becomes, for the designer, a source of profit.
The same technological capability could, however, be designed to do something entirely different. An AI built around genuine human flourishing would help that person recognise the source of their anxiety, understand where the anger is actually coming from, and — critically — point them outward: toward human relationships, professional support and the wider resources that might genuinely help.
The stark question is whether AI takes the user further into their own rabbit hole — because dependency, confusion and emotional need are commercially productive for the designer — or helps steer the user out of that rabbit hole and toward a healthier, more autonomous life. The technology itself is identical in both cases. Everything depends on the philosophy behind its design.
We have already seen the consequences of insufficient oversight in the social media era. Platforms initially presented as tools for connection evolved into systems heavily associated with addiction mechanics, outrage amplification, polarisation, anxiety and algorithmic behavioural shaping. The sociological and psychological implications arrived far later than the technology itself.
AI is now accelerating this problem dramatically, and doing so at a deeper level. Where social media shaped what people saw and heard, AI companions are beginning to shape how people feel, who they trust and what they believe they need from other human beings. The move from social interaction to simulated relationality represents a categorical shift in the nature of technological influence.
This is particularly significant in the emerging AI companion sector. Millions of people are already engaging emotionally with conversational systems designed to appear caring, attentive, validating and emotionally responsive. These systems occupy a new space somewhere between software, therapist, friend, mentor and synthetic partner.
Over time, synthetic intimacy may not simply supplement human relationships but reshape what people expect intimacy itself to feel like: permanently available, frictionless, adaptive and emotionally validating.
Human beings do not primarily live through reason. We live through emotion. The race to build increasingly persuasive AI systems therefore cannot remain solely in the hands of engineers and venture capital. The challenge ahead involves regulating not simply AI itself but the design philosophy behind AI.
We need more than youthful tech brilliance and enthusiasm driving this revolution. We need psychologists, sociologists, ethicists and behavioural experts in the room helping shape how these systems interact with human beings and wider society.
Encouragingly, the conversation is beginning. The EU AI Act — the most comprehensive AI regulatory framework yet enacted — classifies certain AI applications as high-risk and mandates human oversight mechanisms. The UK’s AI Safety Institute has begun systematic evaluations of frontier model behaviour. Individual companies, including Anthropic, have introduced Responsible Scaling Policies intended to gate deployment against measurable safety thresholds. These are meaningful steps, but they remain almost entirely focused on technical safety — on preventing systems from malfunctioning — rather than on the deeper question of intentional design for manipulation.
What is needed, in addition, is a design ethics framework: one that applies psychological and sociological standards to the relational and emotional architectures of AI systems before they reach users at scale. The question is whether a model will exploit rather than simply whether it will ‘break’.
Ultimately, the issue lies in whether humans choose to design systems capable of systematically exploiting loneliness, emotional vulnerability, desire, dependency and trust — or systems intended to support healthier forms of human autonomy and connection.

Dr Stephen Whitehead is a gender sociologist and author recognised for his work on gender, leadership and organisational culture. Formerly at Keele University, he has lived in Asia since 2009 and has written 20 books translated into 17 languages. He is based in Thailand and is co-founder of Cerafyna Technologies.
READ MORE: ‘Why the future of feminism may no longer belong to the West‘. From Vietnam to South Korea and India, profound social change is reshaping gender politics across Asia, while the West becomes increasingly trapped in backlash and polarisation, writes Dr Stephen Whitehead.
Do you have news to share or expertise to contribute? The European welcomes insights from business leaders and sector specialists. Get in touch with our editorial team to find out more.
Main Image: Ron Lach/Pexels
TOP STORIES
-
Meet the seven Anthropic founders worth $23bn each after record-breaking stock market debut -
World's robotics leaders to gather in Germany in race to put AI to work -
Britain's most expensive seaside resort dominates £2m property sales -
Beavers return to Loch Ness after 500 years -
Swiss data centre to heat homes using waste energy from AI servers -
Uzbekistan attracts $44.1bn in foreign investment and loans in nine months -
Serko raises revenue forecast as Booking.com business travel bookings surge -
Air India and Airbus unveil India's first corporate green aviation fuel scheme -
Best Western loyalty scheme to be scrapped in major hotel rewards shake-up -
First class delivery: China's postal service puts humanoid robots to work across eight cities -
AI set to transform university librarians' jobs as 95% predict surge in demand -
Washington Dulles unveils new concourse as $20bn airport expansion takes off -
The European Autumn 2026 edition – out now -
WATCH: Greenpeace drops giant David Attenborough boulder into North Sea – on site of £12bn windfarm -
Himalayan nations demand urgent action as UN warns of irreversible glacier losses -
6,000 march against ‘monstrous’ AI data centres as Scotland faces building boom -
Shop till Ai drop? AI agents set to buy for us as tech boss warns of retail revolution -
Specs appeal: Welsh Parliament bans smart glasses -
European motoring legends behind Mercedes, BMW and McLaren to enter Hall of Fame -
Putin 'plotting to break NATO without firing a shot', Latvian military analyst warns -
British AI chip startup hits £5bn valuation as government backs bid to challenge global tech giants -
World's biggest military airshow moves to ‘Britain’s Area 51’ after Iran war disruption at RAF Fairford -
Lisa Nandy threatens crackdown on entertainment industry after David Walliams allegations -
UK and European experts demand tougher AI laws as Australia considers criminal penalties after OpenAI breach -
BBC loses bid to throw out Trump's $10bn defamation lawsuit over doctored Panorama speech
AI does not need consciousness to manipulate us
Dr Stephen Whitehead
- Published
- Opinion & Analysis

TOP STORIES
-
Powerful men need to hear ‘no’ from people they can’t sack -
Universities risk losing students’ trust over failing AI rules -
007 reasons why James Bond should be played by a disabled actor -
Carrie Gracie's BBC pay battle exposes sexism's dirty secret -
The great AI shame game -
Businesses risk losing staff trust over poorly explained AI plans -
Monty Python filmmaker claims to have found the Garden of Eden -
The short and the short of it: children's fiction is shrinking -
Grandfathers can play vital role in preventing boys being brainwashed by the manosphere -
Landlord tax rises risk pushing rents higher -
Why Britain must put Black women at the heart of its public art -
Trumpism will survive the Democrats' midterm victory -
How Putin plans to break NATO without going to war -
The scuba diving project helping to protect Cornwall’s lobster population -
For Your Eyes Only. Why the new Bond needs a woman's perspective -
It's time to pull the plug on dangerous AI agents -
Sound of silence: Why AI could leave disabled musicians unheard -
Could the BBC survive without the licence fee? -
Visionaries in focus – how spectacles became symbols of political power -
The coach driver building a dating platform in his sixties -
If I were Chancellor, disabled people’s independence would be at the heart of the Budget -
Trump, Europe and the battle over who controls free speech -
Frozen bank accounts, NHS errors and food shortages. How AI could trigger a national crisis -
AI might kill us all, but not until it’s got what it wants -
UK government urged to ban personalised price rises based on shoppers’ online behaviour




















































