The Problem with “Human in the Loop”

“Will my mum do?”

That was Debrah Harding, Managing Director of the MRS, with a wonderfully disarming response at a Zappi Breakfast Club when someone repeated the familiar mantra that AI needs “a human in the loop”.

Everyone laughed. But the joke made an important point.

The issue is not whether a human is involved

The issue has never been whether there is a human involved. The issue is whether the person involved has the expertise to understand the evidence, challenge the assumptions, recognise the limitations of the technology and decide whether the conclusions deserve to influence business decisions.

As Johanna Thompson later summarised the discussion on LinkedIn, perhaps what we really need is not human in the loop, but professional in the loop.

I think that's exactly the conversation our profession now needs to have.

Why the phrase has stopped being enough

I'm writing this wearing my Chair of the Esomar Professional Standards Committee hat because words matter. The language we use shapes expectations, and expectations gradually become professional standards. I think it's time we updated both.

The phrase "human in the loop" has done its job. It helped us through the first wave of AI adoption by reminding us that people remain responsible when AI is involved. But it no longer describes what good professional practice should look like.

In too many organisations, "human in the loop" has become "liability in the loop". AI produces the interview summary, the analysis, the recommendations or the presentation, and somebody is asked to approve it. They may not have designed the workflow, they may not understand how the AI reached its conclusions, and they may not realistically have time to check every step. Yet they are expected to sign their name against the output.

That's not meaningful oversight.

The problem becomes more pressing as AI becomes faster and more capable. If AI produces ten times as much work next year, we cannot expect people to carry out ten times as much careful review. Human judgement doesn't scale at the same pace as machine productivity. A governance model based on somebody checking everything eventually collapses under its own weight.

One of my favourite AI podcasters, Jordan Wilson, has been making a remarkably similar argument. Rather than relying on "human in the loop", he argues for expert-driven loops. The distinction is important. An expert designs the workflow, provides the context, understands the standards against which outputs should be judged and knows when the AI is wrong. Experts actively shape the process rather than simply approving the results.

That resonates strongly with me.

The professional as architect

The role of the professional is changing. We are becoming architects of AI-enabled research, designing systems rather than simply reviewing outputs.

Professional judgement has never been about correcting grammar or spotting the occasional hallucination.

When I review a report, I'm asking much harder questions.

Judgement, not proofreading

• What is this report encouraging our stakeholders and clients to believe?

• What decisions might they make because of these findings?

• Have alternative explanations been considered?

• Are we overstating the certainty of the evidence?

• Would I be comfortable defending these recommendations in front of a board?

Those are questions of judgement, not proofreading.

Take a qualitative study. AI can identify themes, summarise interviews and suggest implications in minutes. That is enormously valuable. But deciding whether those themes genuinely reflect the evidence, whether minority voices have been overlooked, whether important context has been lost, or whether an interesting observation has become an unjustified recommendation still requires professional judgement.

Or consider synthetic audiences. AI can simulate reactions to a concept in seconds. But is a synthetic audience appropriate for this decision? What evidence underpins it? How has it been validated? What are its limitations? Where should we trust it, and where should we be cautious? Again, those are professional decisions.

This is exactly why I think the ICC/Esomar International Code remains so relevant.

What the Code asks of us

The Code doesn't prohibit AI, nor should it. It places responsibility where it has always belonged: with research professionals. It requires us to use appropriate methods, be transparent about how research has been conducted, explain limitations honestly and ensure that our conclusions are supported by evidence. When AI contributes to analysis or reporting, stakeholders and clients should understand both its role and the extent of human oversight.

To me, that means much more than confirming that somebody looked at the output. It means demonstrating that qualified professionals exercised meaningful control over the process.

Debrah Harding calls for professional in the loop. Jordan Wilson argues for expert-driven loops. I think both point towards the same conclusion.

A better question

The future of AI governance isn't about making sure somebody signs off the AI's work. It's about making sure competent professionals design the process, direct the work and take responsibility for the advice ultimately given to stakeholders and clients.

Instead of asking whether there's a human in the loop, let's ask a much better question:

Who is really in control?