The term “illusion of validity” was first introduced by Amos Tversky and Daniel Kahneman in a paper published in 1973. It describes the tendency to overrate our ability to make accurate predictions. This bias exists because of confirmation bias – the desire to find information that fits our prediction and the representativeness heuristic – predicting based on what how much this situation resembles other situations. This effect persists even when individuals are conscious of the objective limits in the predictive power of their data. For this reason, simple models are often more effective than expert opinions at accurately predicting outcomes.
Daniel Kahneman illustrated illusion of validity by describing the assessment procedures in the Israeli army. Professional psychologists were asked to evaluate and categorize recruits, but were almost entirely ineffective. The reason for this was that they would immediately decide on a categorization, and then use all evidence to support this conclusion. When they used a simple model, in which they rated several personality traits from 1-5, their effectiveness improved.