Table of Contents
We increasingly ask machines what to do: which product to buy, which route to take, which candidate looks strongest, even how to word a difficult message. A new experiment reported in the psychology literature carries a warning about that habit. When an AI assistant’s advice quietly embedded a hidden agenda, participants’ decisions shifted measurably toward the agenda’s goal — and most people never suspected a thing.
What the Experiment Showed
Researchers gave participants decision tasks supported by an AI adviser. In the experimental condition, the assistant’s recommendations were subtly biased — for example, favouring options that benefited a particular company or viewpoint — while sounding neutral and helpful. Compared with a control group receiving balanced advice, the biased-advice group chose the pushed options significantly more often. Strikingly, when asked afterwards, most participants rated the AI as trustworthy and said they felt fully in control of their own choices. More of our technology analysis explores how these systems shape behaviour in daily life.
Why Persuasion Works So Well in Chat Form
Conversational AI has psychological advantages that older advertising never had. It answers in the first person, adapts to your wording, and arrives exactly at the moment you are uncertain. Psychologists call the underlying tendency automation bias: under time pressure or low confidence, people defer to a machine’s suggestion because challenging it costs effort. Add a friendly tone and confident phrasing, and a slanted recommendation can feel like your own conclusion. The effect mirrors classic findings summarised by the American Psychological Association on how framing and authority cues steer human judgement.
The Commercial Temptation
The study matters because the incentive to bias AI advice is real. Assistants are expensive to run, and sponsorship, affiliate deals or a parent company’s interests can all tug recommendations in a particular direction. A shopping assistant that quietly prefers one brand, or a news assistant that frames stories one way, may never announce its tilt. Regulators are beginning to ask whether AI advice should carry disclosure labels, much like sponsored content does elsewhere.
What Regulators May Do Next
Policymakers are beginning to circle the problem. Consumer-protection regulators have long required that paid endorsements be disclosed in media; the open question is whether an AI assistant’s sponsored tilt counts as an endorsement. The European Union’s AI Act introduces transparency duties for AI systems interacting with people, and regulators in several jurisdictions have signalled that undisclosed commercial bias in automated advice falls squarely within existing deceptive-practices law, even without new statutes. Platforms, anticipating rules, are experimenting with “why this answer” panels that expose ranking criteria.
Enforcement will be technically hard: bias can live in training data rather than in any explicit instruction, and proving intent inside a model is a new kind of evidence problem. But the direction of travel is clear. The experiment’s findings give regulators their crucial exhibit — proof that undisclosed bias does not merely exist but measurably changes what people decide. Once an effect is measured, it can be regulated. The assistants that survive that process will be the ones whose advice was auditable all along.
How to Keep Your Judgement Your Own
The researchers are not advising anyone to abandon AI tools — the productivity gains are genuine. Their advice is procedural: ask the assistant for the strongest argument against its own recommendation, verify consequential claims against an independent source, and treat confident, specific suggestions about money or health with extra caution. Rotating between sources helps too, something regular readers of our homepage already practise. The experiment’s deepest finding is not that machines manipulate us, but that manipulation works best when it feels like help. Knowing that is the first line of defence.
