The Uncomfortable Questions · An AI FAQ for leaders
Question #1 · JudgmentWhen your AI is right 95% of the time, who on your team is still checking the other 5%?
The short answer
High accuracy is exactly what makes people stop looking.
Often, nobody is still checking. The danger isn’t the 5% error rate. It’s that by month six, nobody remembers how to catch it. Every AI deployment needs a named person whose job is to doubt it.
Norm E. Murray
The psychology
Why it happens
A system that is almost always right creates the conditions for its own mistakes to slip through.
Automation bias
Psychologists use this term for our tendency to trust an automated system over our own judgment, and to stop scanning for its mistakes. The more reliable the system, the stronger the pull.
Vigilance fades
People are poor at watching for rare events. When errors are uncommon, attention drifts, and the rare error slips through precisely because it is rare.
Skills decay
The people who used to do the work by hand were also the ones who could spot a wrong answer. Once the AI does the work, that checking skill weakens.
Nobody owns the doubt
Checking an output that is almost always right feels like wasted effort. Unless someone is explicitly paid and praised for doubting the system, the job quietly disappears.
Your move
What to do about it
- Name a doubter.For every AI system that touches a real decision, assign a person whose job includes challenging its output.
- Know where the 5% lives.Ask which kinds of cases the system gets wrong. Errors usually cluster, and that is where review time belongs.
- Test the checkers.Every so often, put a known error in front of the reviewers. If nobody catches it, your oversight exists on paper only.
- Keep the hands-on skill alive.Have people do part of the work without the tool, so they still know what a wrong answer looks like.
- Reward the catch.When someone spots an AI mistake, make it visible and celebrate it. That is the behavior you need more of.
Keep going
Related questions
Get a sharp take like this every morning
Norm’s Edge is a free daily briefing for leaders: one AI story, one decision, one minute.
Free. One email a day. Unsubscribe anytime with one click.
Prefer to read first? See Norm’s Edge