Base Rate Fallacy: When Stories Hide the Odds
Updated: 21 hours ago
The Setup
Consider an invented hiring example. A candidate is articulate and charismatic. He tells a compelling story about turning around a failing department.
She's impressed.
She hires him on the spot.
Six months later? He's underperforming. The team is in disarray. And she's quietly asking herself the question every leader eventually asks...
How did I get this so wrong?
One possible mistake? She treated a strong story as enough evidence. To judge the decision fairly, we would also need the hiring criteria, comparison data, and conditions he later worked under.
What the Base Rate Fallacy in Leadership Actually Is
The base rate fallacy — also called base rate neglect — is the tendency to ignore general statistical information in favor of vivid, individual-level information.
Kahneman and Tversky studied how people used base rates when predicting a person's profession from a description (1973). This work helped explain why resemblance to a familiar type can outweigh relevant statistical information.
Their findings showed that descriptions could dominate judgments. That does not mean people always ignore statistics. The information and how it is presented matter.
A classic cab example, discussed by Tversky and Kahneman (1982), assumes 85 percent of cabs are green and 15 percent are blue. A witness identifies the cab in an accident as blue. The witness correctly identifies either color 80 percent of the time.
Eighty percent may sound decisive. But the larger number of green cabs matters too.
Under those assumptions, imagine 100 cases with the same color mix. The witness calls 12 of the 15 blue cabs blue, and mistakenly calls 17 of the 85 green cabs blue. That gives 29 "blue" reports, of which 12 are right: 12 ÷ 29, or about 41 percent. The base rate and witness evidence both enter the calculation.
A probability problem can be easier to inspect when you turn it into counts.
Write down the numbers before letting the story decide what they mean.
And as my grandmother would've said... that right there is the whole problem.
Why This Matters for Leadership, Bias, and Belonging
Leadership decisions often bring together interviews, experience, individual cases, and statistical patterns. The job is to weigh them together. An average cannot tell you everything about one person. One person cannot tell you the average.
And it shows up everywhere — including in how organizations approach bias, belonging, and organizational culture work.
Imagine a company copying a training program after hearing one success story. A review of 260 independent samples found positive average effects of diversity training, with smaller effects on behavior and attitudes than on learning and reactions. Benefits were greater when training included awareness and skills, ran over time, and sat alongside other initiatives (Bezrukova et al., 2016). That review does not say most single-session bias training fails.
A leader fires one underperforming employee from a particular background and quietly stops hiring people from that background. One vivid story. A lifetime of statistical reality overwritten in an afternoon.
This is how the base rate fallacy turns into bias.
This is how bias gets dressed up as "experience."
Where the Base Rate Fallacy Lives
Here are three invented examples. Their lesson is to ask for comparison data, not to assume what that data will show.
The church-planting example. An organization sees one church grow rapidly and wants to copy its approach. Before changing the whole training model, ask how often similar churches grew, over what period, and under what conditions.
Include churches that closed or grew slowly in the comparison. Define what success means. A large attendance count, a stable budget, and a healthy community are different outcomes. One success story cannot settle them all.
The school-district example. A district considers a curriculum after one school's scores rise. Ask for results across comparable schools, including those with weaker outcomes. Check how students were selected and what else changed. The example is hypothetical; no curriculum success rate is assumed here.
The CEO example. One famous college dropout should not settle a hiring policy. BLS data for 2024 show higher median weekly earnings and lower unemployment at higher education levels among adults 25 and older. Earnings cover full-time wage and salary workers (BLS, 2025). Those averages do not establish causation, leadership ability, or an individual applicant's fit.
In every case, the leader wasn't unintelligent. The leader was human.
Representativeness means judging by resemblance to a familiar type (Kahneman & Tversky, 1973). Similarity may be useful evidence. It is not a substitute for the relevant base rate.
A good story deserves a hearing. It still has to share the room with the math.
Faith and Leadership: Read the Wisdom in Context
Here's where it gets interesting.
These biblical passages offer wisdom about judgment. They are not lessons in Bayesian statistics. The connection below is a modern application, not the original scientific meaning of the text.
Proverbs 18:17 warns that the first account can seem right until another person examines it. Its focus is hearing a claim critically, not calculating a population rate. Read the passage in the ESV. Proverbs 18:17 (ESV)
A leadership application: hear the compelling account, then make room for questions, other evidence, and people affected by the decision.
Ecclesiastes 7:10 challenges the assumption that earlier days were better. It questions nostalgia; it does not provide a statistical test. Read the passage in the ESV. Ecclesiastes 7:10 (ESV)
A leadership application: examine what a memory leaves out before calling the past a model for the future.
Respect the context of the ancient text. Then state clearly how you are applying it today.
Trauma-Informed Lens
Here's where this gets personal.
A trauma-informed discussion should protect safety and agency (SAMHSA, n.d.). Do not treat a person's caution as proof that they cannot reason with statistics.
If someone describes harm, hear the account before reaching for an average. Population data cannot prove that this particular person or workplace is safe.
Ask whether the comparison group fits the current case. Historical averages can also reflect unfair conditions. A base rate deserves inspection, just as a personal story does.
Two useful checks for the meeting:
One. Explain where the numbers came from, who is included, and what is missing. Make room for questions about the data and the decision.
Two. Give people a way to raise concerns without disclosing a trauma history. Their disagreement may reveal a real limit in your evidence. These are leadership practices, not a diagnosis or treatment.
What the Research Says About Mitigation
Research on probability tasks shows that how information is presented and taught can improve reasoning. That supports trying better tools. It does not establish that these meeting practices rewire a brain or solve every leadership bias.
1. Ask the question. Out loud. Every time.
Before any major decision, pause and ask: What is the base rate?
What does the broader data say about success rates for this type of hire, program, initiative, investment?
Gigerenzer and Hoffrage (1995) found that natural-frequency formats helped participants solve Bayesian problems. Use connected counts with a shared total, as in the cab example above. Merely changing any percentage into "x out of 100" is not the whole method.
Practical shift: Before your next major decision, research the base rate of success for similar initiatives. Write it on the whiteboard before the discussion starts. Let it stare at you the whole meeting.
2. Separate the story from the statistics.
Give the case story and the comparison data separate space in the decision record. Then examine how each should affect the decision. This is a practical process recommendation, not a claim that one study proved a universal meeting order.
The fix? Structurally separate them in your decision processes.
Practical shift: Review the comparison data, hear the individual case, and record where the case fits or differs. Consider reasons the average may not apply. Keep both available for scrutiny.
3. Use structured decision-making frameworks.
Sackett and colleagues' updated review reports that structured interviews had the highest average job-performance predictive validity among the methods compared (2023). Results vary across settings. This supports careful interview design; it does not prove that structure works by correcting base-rate neglect.
Practical shift: Use clear, job-relevant criteria and the same core questions for each candidate. Review evidence about the individual. Check whether past comparisons reflect unfair barriers before letting them shape a new decision.
4. Build statistical literacy as a leadership competency.
In two studies, Sedlmeier and Gigerenzer (2001) compared a tutorial on frequency representations with training in Bayes's rule. Frequency training showed stronger immediate learning and retention. These were probability tasks, not a direct test of better workplace leadership.
Statistical literacy is not just for analysts.
It's a leadership skill.
Practical shift: Include basic statistical reasoning in your leadership development programs. Train your team to ask "What is the base rate?" with the same reflex they use for "What's the budget?"
The Leadership Call
Stories move us.
Data grounds us.
Effective leaders need both.
The base rate fallacy doesn't mean abandon stories. It means never let a single story replace the broader pattern. The most compelling testimony in the room should complement the data — not override it.
When a leader ignores base rates, they're not being visionary.
They're being statistically reckless.
And the people who pay the price? The teams. The organizations. The communities that bear the consequences of decisions built on beautiful stories and broken math.
So the next time someone walks in with a spectacular case study, an inspiring testimony, or a single data point that feels like a mandate... pause.
Ask the uncomfortable question.
What is the base rate?
What happens to most people, most organizations, most initiatives in this category?
That question won't make you less inspiring.
It will make you more trustworthy.
Use the broader pattern to inform the individual case.
Use the individual case to test whether the comparison fits.
That's how leaders make decisions that hold up long after the story fades.
Keep reading: Pattern-Seeking Bias: When Patterns Mislead Us and Barnum Effect in Leadership: When Feedback Says Little. Or browse the whole Bias Field Guide.
Seeing this in your own organization? See how I work with leaders.
References
Bezrukova, K., Spell, C. S., Perry, J. L., & Jehn, K. A. (2016). A meta-analytical integration of over 40 years of research on diversity training evaluation. Psychological Bulletin, 142(11), 1227–1274. Diversity training review (2016)
Gigerenzer, G., & Hoffrage, U. (1995). How to improve Bayesian reasoning without instruction: Frequency formats. Psychological Review, 102(4), 684–704. Frequency formats study (1995)
Kahneman, D., & Tversky, A. (1973). On the psychology of prediction. Psychological Review, 80(4), 237–251. Psychology of prediction (1973)
U.S. Bureau of Labor Statistics. (2025, May). Education pays, 2024. Career Outlook. Education pays, 2024 (BLS)
Substance Abuse and Mental Health Services Administration. (n.d.). Trauma-Informed Approaches and Programs. SAMHSA guidance
Sackett, P. R., Zhang, C., Berry, C. M., & Lievens, F. (2023). Revisiting the design of selection systems in light of new findings regarding the validity of widely used predictors. Industrial and Organizational Psychology, 16(3), 283–300. Selection-system review (2023)
Sedlmeier, P., & Gigerenzer, G. (2001). Teaching Bayesian reasoning in less than two hours. Journal of Experimental Psychology: General, 130(3), 380–400. Bayesian reasoning training (2001)
Tversky, A., & Kahneman, D. (1982). Evidential impact of base rates. In D. Kahneman, P. Slovic, & A. Tversky (Eds.), Judgment under Uncertainty: Heuristics and Biases. Cambridge University Press. Base-rate evidence chapter (1982)
