Why are we overconfident in our predictions?

The 

Illusion of Validity

, explained.
Bias

What is the Illusion of Validity?

The illusion of validity is a cognitive bias that describes our tendency to be overconfident in the accuracy of our judgements, specifically in our interpretations and predictions regarding a given data set.

Where this bias occurs

Teachers often believe that they can accurately predict how well a student will do in their course based on their past performance at the school. While a teacher might be very confident in their predictions about a certain student, there may be things going on behind the scenes that lead to drastically different outcomes. For example, if a student with a pristine record begins to develop symptoms of a mental illness, such as depression, or suddenly finds themselves in the midst of their parents’ messy divorce, they may no longer make straight-As, and may even begin to act out. On the other hand, a student who typically gets lower grades may realize that the college program they want to get into is quite competitive, and therefore decide to pull their act together and double-down on their studying. In both cases, a teacher might have predicted the student’s performance in their class based on a pattern that ended up wrong.

Sources

  1. Penn, A. (2019). Illusion of Validity: Think You Make Good Predictions? Shortform. https://www.shortform.com/blog/illusion-of-validity/
  2. Kahneman D. and Tversky, A. (1973). On the Psychology of Prediction. Psychology Review. 80(4), 237-251. doi: 10.1037/h0034747
  3. See 2
  4. “Major Depression”. The National Institute of Mental Health. https://www.nimh.nih.gov/health/statistics/major-depression.shtml
  5. See 2
  6. See 2
  7. See 2
  8. See 2
  9. Kahneman, Daniel (2011). “Don’t Blink! The Hazards of Confidence”. The New York Timeshttps://www.nytimes.com/2011/10/23/magazine/dont-blink-the-hazards-of-confidence.html
  10. Gilles, F., Gressens, P., Dammann, O., & Leviton, A. (2018). Hypoxia-ischemia is not an antecedent of most preterm brain damage: the illusion of validity. Developmental Medicine and Child Neurology60(2), 120–125. doi: 10.1111/dmcn.13483

About us

We are the leading applied research & innovation consultancy

Our insights are leveraged by the most ambitious organizations

Image

I was blown away with their application and translation of behavioral science into practice. They took a very complex ecosystem and created a series of interventions using an innovative mix of the latest research and creative client co-creation. I was so impressed at the final product they created, which was hugely comprehensive despite the large scope of the client being of the world's most far-reaching and best known consumer brands. I'm excited to see what we can create together in the future.

Heather McKee

BEHAVIORAL SCIENTIST

GLOBAL COFFEEHOUSE CHAIN PROJECT

OUR CLIENT SUCCESS

$0M

Annual Revenue Increase

By launching a behavioral science practice at the core of the organization, we helped one of the largest insurers in North America realize $30M increase in annual revenue.

0%

Increase in Monthly Users

By redesigning North America's first national digital platform for mental health, we achieved a 52% lift in monthly users and an 83% improvement on clinical assessment.

0%

Reduction In Design Time

By designing a new process and getting buy-in from the C-Suite team, we helped one of the largest smartphone manufacturers in the world reduce software design time by 75%.

0%

Reduction in Client Drop-Off

By implementing targeted nudges based on proactive interventions, we reduced drop-off rates for 450,000 clients belonging to USA's oldest debt consolidation organizations by 46%

Notes illustration

Eager to learn about how behavioral science can help your organization?