Heuristics May Bias Judicial Decision-Making

Chief Justice John Roberts used a now-famous umpire analogy during his confirmation hearing in September 2005 to describe the role of a judge, or justice, as he saw it: “Judges are like umpires. Umpires don’t make the rules they apply them.”^[2] This view of a judge’s role, which has been heavily debated,^[3] paints a picture of the judge as a neutral and impartial arbiter. It brings to mind the image of a judge who decides disputes entirely rationally and consistently, based only on a case’s objective merits.^[4]

However, a growing amount of research suggests that this picture of a judge—as an impartial and wholly rational arbiter—is not quite correct, or at minimum, incomplete.^[5] Rather, empirical evidence suggests that judges are in fact “constrained by the boundaries of human cognition.”^[6]

For instance, Peer and Gamliel reviewed several studies whose findings suggest that “irrelevant factors that should not affect judgment might cause systemic and predictable biases in judges’ decision-making processes in a way that could be explained using known cognitive heuristics and biases.”^[7] Rachlinski and Wistrich, the latter a former federal judge himself, conducted research involving one thousand judges and found that judicial decisions run the risk of being less consistent than one might think, due in part to a “susceptibility to framing … caused by uncritical reliance on heuristic processing, which can be exacerbated by time pressure.”^[8]

The extent to which judges rely on intuition, and are thus susceptible to heuristic-based persuasion, can be predicted fairly well by cognitive reflection tests (CRTs).^[9]The purpose of the CRT is to “test[] a respondent’s ability to suppress intuition in favor of deliberation in a setting where intuition is misleading.”^[10] Essentially, a CRT contains questions that are simple but require some deliberate processing; a reliance on mental shortcuts and intuition will typically lead to a wrong—but a predictably wrong—answer.^[11]

For example: A bat and a ball cost $1.10 in total. The bat costs $1.00 more than the ball. How much does the ball cost? ^[12] Most people immediately answer that the ball costs 10 cents. ^[13] Intuitively this question seems obvious, but the point is that here, intuition leads to the wrong answer. If the ball costs 10 cents, the bat must cost $1.10, leading to a total of $1.20. Instead, getting to the correct answer (the ball costs 5 cents) requires suppression of the initial intuitive response. A typical CRT has three of these sorts of questions on it. ^[14]

Studies on judges showed that “judges performed about as well as the most educated adults on the CRT” ^[15]—not terribly well, answering on average just over 1 out of 3 questions correctly. ^[16]

A low CRT score suggests that judges, when making decisions in a generalized setting at least, often make intuitive decisions when they should be reflective.

Of course, a judge may make decisions differently in court than when making ordinary, non-legal decisions. And, judges can be forgiven for being bad at math. ^[17]

But, the CRT is remarkably robust, and it does actually predict fairly well the extent to which those who score well (engineers), or poorly (judges), use deliberate or intuitive reasoning in other contexts. ^[18] In recent years, studies have been done showing that judges are inclined to rely on intuitive judgments and only have a mild disposition toward overriding them.