Type II Error
What is a Type II Error?
A type II error occurs when a statistical test fails to detect a real effect, leading researchers to incorrectly retain the null hypothesis. In other words, it’s a false negative—the test misses a true relationship or difference that actually exists.
The Basic Idea
We’ve all heard that drinking a coffee late will make it harder to fall asleep, but does caffeine really impact our sleep? You’re not sure, so you decide to conduct an at-home experiment.
Like any good experiment, you start with two potential hypotheses:
Null hypothesis (H0): Drinking coffee in the evening has no effect on your sleep
Alternative hypothesis (H1): Drinking coffee in the evening negatively impacts your sleep
The first day, you don’t drink any coffee and go to bed at 10:00 pm. You fall asleep quickly and, according to your sleep-tracking app, achieve an 87% sleep score. For two days after, you drink coffee at 7:00 pm and go to bed at 10:00 pm. You’re still able to fall asleep quickly, and your sleep score doesn’t vary much from the first day. Given the evidence, you retain the null hypothesis that coffee doesn’t affect your sleep and reject the alternative hypothesis.
However, in reality, the caffeine does impact your sleep—the experiment you designed just didn’t detect the effect. It was only a small sample size of one participant, you only compared one day without coffee to two days with coffee, and other factors—like how active you were those days, fatigue, stress, or screen time—all may have impacted your sleep quality. Even though you tried to control variables by maintaining a consistent bedtime and attempted to measure your sleep quality in an unbiased way with an app, you have failed to reject the null hypothesis.
This type of error is known as a type II error. Type II errors occur when a null hypothesis is incorrectly retained and a conclusion is made that there is insufficient evidence to suggest that a variable (in this instance, coffee) has a particular effect on an outcome (in this instance, sleep quality). A true effect is missed, resulting in a “false negative.”1 Your conclusion from your at-home experiment might result in caffeine intake late in the evening, and you may eventually feel the repercussions when you continuously wake up groggy. But in more serious contexts, type II errors can have higher stakes. Imagine if a medical trial testing the effect of a new drug treatment for depression falsely concluded that the drug had no effect on patient outcomes? The drug would be dismissed, never making it out of the clinical trial stage, delaying critical treatments.
We must watch our own language. For example, ‘Type I error’ and ‘Type II error’ are meaningless and misleading terms. Instead, try "chance of a false alarm" and "a missed opportunity.
— Deborah Rumsey, American statistician and educator, in her paper “Assessing Student Retention of Essential Statistical Ideas: Perspectives, Priorities, and Possibilities"2
About the Author
Emilie Rose Jones
Emilie currently works in Marketing & Communications for a non-profit organization based in Toronto, Ontario. She completed her Masters of English Literature at UBC in 2021, where she focused on Indigenous and Canadian Literature. Emilie has a passion for writing and behavioural psychology and is always looking for opportunities to make knowledge more accessible.
