Statistical Significance
What is Statistical Significance?
Statistical significance is a measure used in hypothesis testing to determine if the observed results in a study are unlikely to have occurred due to chance alone, typically assessed using a p-value threshold (e.g., p < 0.05). It provides confidence that the effect or relationship observed in the data is real and not random. Understanding statistical significance is crucial for making informed, evidence-based decisions in research and data analysis.
The Basic Idea
How many times in life have you noticed a strange phenomenon? Maybe the same song you were just humming came on the radio, you bumped into someone from your hometown in a completely different part of the world, or you’ve noticed that every time you have oatmeal for breakfast, you hit all the green lights on the way to work. While each of these could be a coincidence, wouldn’t it be nice to have a way to figure out whether or not they truly were? Or at least determine the odds that these events were due to chance alone? While it may not be entirely possible to calculate whether each of these personal situations was completely coincidental, the concept of assessing the likelihood that there is a force besides chance at play, known as statistical significance, can be applied to many other phenomena in the world.
Statistical significance is used in many types of research to determine whether the results of a study are likely to be due to a specific factor rather than occurring by chance. Researchers use statistical tests to assess this, including calculating a p-value. The p-value is a measure that helps determine the likelihood of observing the obtained results if the null hypothesis (which suggests no effect or no difference between groups) is true. Basically, the p-value answers the question: “Is this result unlikely to be due to random chance?” If the p-value falls below a predetermined threshold, the result is deemed statistically significant. Usually, the threshold of 0.05 is used (meaning you’re looking for a p-value of 0.05 or lower) because it indicates less than a 5% probability that the observed results are due to chance.1
However, statistical significance only speaks to the presence of an effect—not its magnitude or practical importance. Findings from a study could be statistically significant but have a negligible effect in real-world applications, either because the methodology was flawed, the effect size was too small, or some other factor. Although researchers should consider statistical significance in any study, many other considerations are needed to understand the practical implications of statistical results.2
“A statistical analysis, properly conducted, is a delicate dissection of uncertainties, a surgery of suppositions.”
— M. J. Moroney, statistician and author of Facts from Figures.
About the Author
Annika Steele
Annika completed her Masters at the London School of Economics in an interdisciplinary program combining behavioral science, behavioral economics, social psychology, and sustainability. Professionally, she’s applied data-driven insights in project management, consulting, data analytics, and policy proposal. Passionate about the power of psychology to influence an array of social systems, her research has looked at reproductive health, animal welfare, and perfectionism in female distance runners.
