The Uncomfortable Questions · An AI FAQ for leaders
Question #20 · GovernanceIf an AI recommendation goes wrong, who pays the price: the person who followed it, or the person who overruled it?
The short answer
Fix the incentive and you fix half your AI risk.
In most organizations, overruling the AI and being wrong is career-limiting, while following it and being wrong is forgivable. I call this the Override Penalty. It trains people to defer, even when their judgment is better.
Norm E. Murray
The psychology
Why it happens
No one writes the Override Penalty into a policy. It forms on its own, from how bad outcomes get judged after the fact and from who has cover when they land.
The default carries no blame
Kahneman and Miller’s norm theory shows that people react more strongly to a bad outcome that follows a departure from the default. Once the AI’s answer is the default, overruling it is the departure, and the person who did it stands alone.
Results get judged, not reasoning
Baron and Hershey named it outcome bias: we rate a decision by how it turned out, not by how sound it was at the time. An override that fails looks reckless, however good the thinking behind it.
The safe choice beats the best choice
Gerd Gigerenzer calls it defensive decision making: choosing the option that protects you rather than the one you believe is right. “The model said so” is the best protection a professional has ever been handed.
You get what you actually reward
Steven Kerr’s classic paper is titled “On the Folly of Rewarding A, While Hoping for B.” Leaders ask for independent judgment, then punish it the first time it misses. People learn from the punishment, not the speech.
Your move
What to do about it
- Look at what happened to the last override.Find someone who overruled the AI and was wrong, and someone who followed it and was wrong. If their consequences differ, you have found your penalty.
- Review the reasoning, not just the result.In every post-mortem, ask whether the call was sound on what was known at the time, whichever way it went.
- Track both kinds of error.Log bad outcomes from following the AI with the same care as bad outcomes from overruling it. Most organizations only see one column.
- Ask for a reason in both directions.On high-stakes decisions, require a short note for accepting the recommendation as well as for rejecting it. Deferring is a decision too.
- Back a good override in public.When someone overrules the AI with sound reasoning and the outcome goes against them, a senior leader should say so out loud. People will remember that longer than any policy.
Keep going
Related questions
Get a sharp take like this every morning
Norm’s Edge is a free daily briefing for leaders: one AI story, one decision, one minute.
Free. One email a day. Unsubscribe anytime with one click.
Prefer to read first? See Norm’s Edge