The Uncomfortable Questions · An AI FAQ for leaders
Question #22 · GovernanceCan your board explain how your most important AI system makes decisions? Should it be able to?
The short answer
Boards don’t need to understand the math.
They do need to understand what the system optimizes for, where it’s likely to fail, and who notices when it does. If no director can answer those three questions, the board is governing on trust, not oversight.
Norm E. Murray
The psychology
Why it happens
Technical complexity gives boards a reason to defer, and deferring feels like delegation when it is really a gap in oversight.
Feeling informed is not understanding
Psychologists Leonid Rozenblit and Frank Keil named the illusion of explanatory depth: we believe we understand complex systems until we try to explain them step by step. A board that has seen a demo can feel informed without being able to answer a single hard question.
The board sees what it is shown
Management and vendors know far more about an AI system than directors do, a classic information asymmetry. Without deliberate effort, the board only sees what the people it oversees choose to show it.
Every system optimizes for something
Every model has an objective function: the one target it is built to maximize, such as conversion, approval rate or cost per case. If the board never asks what that target is, it cannot see the trade-offs the system makes on the company’s behalf.
Everyone assumes someone else is watching
When oversight is shared across the board, a committee and management, each group assumes another is watching. Psychologists call it diffusion of responsibility, and it is how failures go unnoticed until they are public.
Your move
What to do about it
- Ask the three questions.For your most important AI system: what does it optimize for, where is it likely to fail, and who notices when it does?
- Require a one-page explanation.Management should describe each critical system in plain language a director can challenge.
- Name an owner.Make one executive accountable to the board for each critical system’s behavior, not just its uptime.
- Review failures, not demos.Ask to see the cases the system got wrong and what happened next.
- Build director fluency.Schedule regular briefings so at least some directors can ask informed follow-up questions.
Keep going
Related questions
Get a sharp take like this every morning
Norm’s Edge is a free daily briefing for leaders: one AI story, one decision, one minute.
Free. One email a day. Unsubscribe anytime with one click.
Prefer to read first? See Norm’s Edge