Why are we overconfident in our predictions?

Illusion of Validity

, explained.
Bias

What is the Illusion of Validity?

The illusion of validity is a cognitive bias that describes our tendency to be overconfident in the accuracy of our judgements, specifically in our interpretations and predictions regarding a given data set.

Where this bias occurs

Teachers often believe that they can accurately predict how well a student will do in their course based on their past performance at the school. While a teacher might be very confident in their predictions about a certain student, there may be things going on behind the scenes that lead to drastically different outcomes. For example, if a student with a pristine record begins to develop symptoms of a mental illness, such as depression, or suddenly finds themselves in the midst of their parents’ messy divorce, they may no longer make straight-As, and may even begin to act out. On the other hand, a student who typically gets lower grades may realize that the college program they want to get into is quite competitive, and therefore decide to pull their act together and double-down on their studying. In both cases, a teacher might have predicted the student’s performance in their class based on a pattern that ended up wrong.

Related Biases

Sources

  1. Penn, A. (2019). Illusion of Validity: Think You Make Good Predictions? Shortform. https://www.shortform.com/blog/illusion-of-validity/
  2. Kahneman D. and Tversky, A. (1973). On the Psychology of Prediction. Psychology Review. 80(4), 237-251. doi: 10.1037/h0034747
  3. See 2
  4. “Major Depression”. The National Institute of Mental Health. https://www.nimh.nih.gov/health/statistics/major-depression.shtml
  5. See 2
  6. See 2
  7. See 2
  8. See 2
  9. Kahneman, Daniel (2011). “Don’t Blink! The Hazards of Confidence”. The New York Times. https://www.nytimes.com/2011/10/23/magazine/dont-blink-the-hazards-of-confidence.html
  10. Gilles, F., Gressens, P., Dammann, O., & Leviton, A. (2018). Hypoxia-ischemia is not an antecedent of most preterm brain damage: the illusion of validity. Developmental Medicine and Child Neurology, 60(2), 120–125. doi: 10.1111/dmcn.13483

About the Authors

A man in a blue, striped shirt smiles while standing indoors, surrounded by green plants and modern office decor.

Dan Pilat

Dan is a Co-Founder and Managing Director at The Decision Lab. He is a bestselling author of Intention - a book he wrote with Wiley on the mindful application of behavioral science in organizations. Dan has a background in organizational decision making, with a BComm in Decision & Information Systems from McGill University. He has worked on enterprise-level behavioral architecture at TD Securities and BMO Capital Markets, where he advised management on the implementation of systems processing billions of dollars per week. Driven by an appetite for the latest in technology, Dan created a course on business intelligence and lectured at McGill University, and has applied behavioral science to topics such as augmented and virtual reality.

A smiling man stands in an office, wearing a dark blazer and black shirt, with plants and glass-walled rooms in the background.

Dr. Sekoul Krastev

Sekoul is a Co-Founder and Managing Director at The Decision Lab. He is a bestselling author of Intention - a book he wrote with Wiley on the mindful application of behavioral science in organizations. A decision scientist with a PhD in Decision Neuroscience from McGill University, Sekoul's work has been featured in peer-reviewed journals and has been presented at conferences around the world. Sekoul previously advised management on innovation and engagement strategy at The Boston Consulting Group as well as on online media strategy at Google. He has a deep interest in the applications of behavioral science to new technology and has published on these topics in places such as the Huffington Post and Strategy & Business.

About us

We are the leading applied research & innovation consultancy

Our insights are leveraged by the most ambitious organizations

Image

“

I was blown away with their application and translation of behavioral science into practice. They took a very complex ecosystem and created a series of interventions using an innovative mix of the latest research and creative client co-creation. I was so impressed at the final product they created, which was hugely comprehensive despite the large scope of the client being of the world's most far-reaching and best known consumer brands. I'm excited to see what we can create together in the future.

Heather McKee

BEHAVIORAL SCIENTIST

GLOBAL COFFEEHOUSE CHAIN PROJECT

OUR CLIENT SUCCESS

$0M

Annual Revenue Increase

By launching a behavioral science practice at the core of the organization, we helped one of the largest insurers in North America realize $30M increase in annual revenue.

0%

Increase in Monthly Users

By redesigning North America's first national digital platform for mental health, we achieved a 52% lift in monthly users and an 83% improvement on clinical assessment.

0%

Reduction In Design Time

By designing a new process and getting buy-in from the C-Suite team, we helped one of the largest smartphone manufacturers in the world reduce software design time by 75%.

0%

Reduction in Client Drop-Off

By implementing targeted nudges based on proactive interventions, we reduced drop-off rates for 450,000 clients belonging to USA's oldest debt consolidation organizations by 46%

Let's chat

Read Next

Illusory Correlation

Why do we think some things are related when they aren’t?

Why do we think some things are related when they aren’t?

Learn More

Illusory Truth Effect

Why do we believe misinformation more easily when it’s repeated many times?

Why do we believe misinformation more easily when it’s repeated many times?

Learn More

In-group Bias

Why do we treat our in-group better than we do our out-group?

Why do we treat our in-group better than we do our out-group?

Learn More
See All
Notes illustration

Eager to learn about how behavioral science can help your organization?

Contact us