The Uncomfortable Questions · An AI FAQ for leaders
Question #21 · GovernanceIn your organization, is “human in the loop” a safeguard or a signature?
The short answer
That’s not oversight. It’s a signature on a decision someone else made.
A human in the loop only protects you if that person has the time, information and authority to say no. Too often they’re rubber-stamping hundreds of outputs a day. Ask your teams how often the human actually changes the outcome.
Norm E. Murray
The psychology
Why it happens
Nobody sets out to turn a reviewer into a rubber stamp. It happens on its own, when a person is asked to watch a reliable system at high volume with no real power to stop it.
Reliable systems get waved through
Raja Parasuraman’s research on automation complacency shows that the more dependable a system seems, the less closely people monitor it. Reviewers stop checking long before anyone notices they have stopped.
Attention fades on repetitive checks
Psychologists call it the vigilance decrement: the ability to spot rare problems drops the longer someone monitors a stream of mostly correct items. Hundreds of approvals a day is exactly that stream.
Everyone assumes someone else is checking
Darley and Latané named it diffusion of responsibility: when responsibility is shared, each person feels less of it. The reviewer trusts the model’s builders, and the builders trust the reviewer.
The human absorbs the blame
Researcher Madeleine Clare Elish calls it the moral crumple zone: the person nearest the system takes the blame for failures they had little power to prevent. A signature without authority is a liability, not a control.
Your move
What to do about it
- Measure how often the human says no.Ask each team for the share of AI outputs its reviewers changed or rejected last quarter. A number near zero is a warning, not a sign of quality.
- Size the work to the review.If one person signs off on hundreds of items a day, the loop is decorative. Cut the volume, sample instead, or add reviewers on the decisions that matter most.
- Give reviewers what they need to judge.Show them the inputs, the reasoning and the confidence behind each output, not just the answer. Nobody can catch an error they cannot see.
- Put the authority to say no in writing.Make clear who can stop a decision and protect them when they do. Otherwise the Override Penalty will teach them to sign.
- Test the loop on purpose.Slip known errors into the review queue from time to time and see how many get caught. That tells you more than any process diagram.
Keep going
Related questions
Get a sharp take like this every morning
Norm’s Edge is a free daily briefing for leaders: one AI story, one decision, one minute.
Free. One email a day. Unsubscribe anytime with one click.
Prefer to read first? See Norm’s Edge