Randomized Controlled Trial

Imagine that you wanted to know how effective an antidepressant is. While you could give the medication to an entire group of people who experience depression, it would be difficult to accurately measure its effectiveness because you wouldn’t be able to compare the results with a group of people who hadn’t taken the medication. 

As such, you may design a clinical experiment where one group of people receive the antidepressant, and the other group—the control group—would not. In a randomized controlled trial, participants are randomly selected to be in the treatment (or experimental) group or the control group to avoid selection bias. By comparing the outcomes of the treatment group to the control group, you would be able to assess the effectiveness of the antidepressant more accurately.

Randomization

The hope in a randomized controlled trial is that the only significant difference between the people in the control and treatment groups is whether they receive the treatment or intervention being studied. Due to the random allocation of people into the groups, participant characteristics such as age, race, and gender are usually balanced, which allows researchers to attribute any difference in outcome to the intervention. 

There are a few different ways to randomize participants:¹

• Simple randomization: the most straightforward way to randomize people into groups, using tools like random number generators to assign all participants
• Block randomization: assigning participants into blocks first, and then assigning each block to a group. If you have 100 participants, you could divide them into four blocks of 25 each, then assign blocks 1 and 3 to the treatment group and blocks 2 and 4 to the control group.
• Stratified Randomization: assigning participants into different groups based on characteristics that could affect the results. For example, if you want an equal number of men and women in your treatment and control group, you would use sex to sort participants—assigning an even amount to each group

Randomization is done with the aim of balancing known and unknown confounders across groups, but it's not guaranteed that all characteristics will be perfectly balanced, especially in smaller trials.

Control group: A group of participants that do not receive the intervention that is being studied, whose outcomes are compared to a treatment group

Treatment group: A group of participants that receives the intervention that is being studied.

Selection bias: A flaw in a research study where there is bias in the sample selection process, such as allocating people into a control or treatment group. For example, if researchers ask people to volunteer for each group, it could lead people with specific characteristics to volunteer and skew the outcomes.³

Single-blind experiment: An experimental design in which the participants do not know whether they are receiving the treatment or a placebo, but the researchers do.

Double-blind experiment: An experimental design where both the participants and the researchers are unaware as to what condition each of the participants are in.

Placebo: A faux substance or intervention used in a blind experiment so that participants are unaware if they are receiving the intervention. Using a placebo in an experiment allows researchers to identify whether outcomes are psychological effects of thinking someone received treatment or actually due to receiving the intervention. 

Power (Statistical Power): ensuring that you have enough people in both the control and treatment groups to see a statistical association between treatment and outcome. If the effect of a treatment is minimal, you will require a larger sample size to infer correlation.

Confounding Variable: an extraneous variable that is not appropriately controlled in a study. The presence of a confounding variable can create a false impression of a cause-and-effect relationship between treatment and outcome. For example, if a researcher is trying to see if exercise leads to weight loss, a confounding variable would be diet. 

Response bias: our tendency to provide inaccurate, or even false, answers to self-report questions, such as those asked on surveys or in structured interviews.

It’s difficult to pinpoint exactly when randomized controlled trials began, but James Lind, a Scottish physician, is often credited as the first person to conduct a controlled trial in 1754. 

During the eighteenth century, the British were involved in the War of Austrian Succession against France and Spain, with many men at sea in battle. At this time, more sailors were dying of scurvy than actually dying in combat. People didn’t know what was causing the illness, but Lind was determined to find out. During a voyage, Lind divided 12 sick sailors into six different pairs, and provided them each with different interventions. Notably, one pair received oranges and lemons—and they were the only sailors whose health improved. From this experiment, Lind concluded that citrus fruit helped to cure scurvy.⁴

While Lind’s sample size looks nothing like today’s clinical trials, and we can’t be sure if the pairs were randomly assigned, it was one of the first times that various conditions (including a control condition) were used in testing the effectiveness of an intervention. Throughout the 19th century, the use of control groups became more popular, but researchers were not yet aware of the importance of randomizing the groups.

Randomized controlled trials, as we know them today, began to take shape in the mid-20th century. The British Medical Research Council’s experiment using streptomycin as a treatment for pulmonary tuberculosis (1948) is often cited as the first real randomized controlled trial. In this study, patients with pulmonary tuberculosis were randomly assigned to either a control group, that would only be treated through bed rest (at the time, the current standard treatment for pulmonary tuberculosis), or a treatment group, where they would receive streptomycin and bed rest. 

The researchers had no prior knowledge as to who would receive which treatment until they were given an envelope right before seeing a patient, and the patients were unaware that they were in a trial, minimizing the influence of bias (and maximizing the unethicalness). 

The researchers were more accurately able to draw a relationship between receiving streptomycin and improved health outcomes. With each group being made up of around 55 patients, only four patients in the treatment group died within six months of receiving treatment compared to 15 that were assigned only to bed rest.⁵

After the 1948 experiment, randomized controlled trials gained popularity and quickly became the gold-standard in most fields, especially in medical research.

James Lind⁶

A Scottish physician who is credited with conducting the first randomized controlled trial in 1754, where he studied the effects of six different interventions to cure scurvy in sailors. He found that eating citrus fruit helped to diminish symptoms of scurvy. Throughout his career, Lind continued to advocate for the health of seamen, outlining best hygiene practices and environmental factors that would lead to better health when at sea. 

Archie Cochrane⁷

Often referred to as the father of evidence-based medicine, Cochrane was a Scottish medical researcher who believed there was a lack of scientific evidence backing medical interventions in the 20th century. He conducted one of the first clinical trials in 1941, exploring how yeast could reduce starvation in prisoners.⁸ Throughout his career, Cochrane was a strong advocate for the need for proper assessment of reliable evidence in medical care. 

Austin Bradford Hill⁹

A British scientist who began his career in economics but later transitioned into medical research, Hill pioneered the randomization component of randomized controlled trials. He proposed the randomization of subjects into treatment and control groups in the British Medical Research Council’s 1948 study on the efficacy of streptomycin in treating pulmonary tuberculosis. This trial set the standard for future control trials. Later, Hill worked with Richard Doll to demonstrate the causal relationship between smoking and lung cancer. 

Randomized controlled trials are considered the gold standard in clinical research and are required by regulatory bodies, such as the The Food and Drug Administration, to approve new treatments for the market. 

They are effective as they help to isolate the effects of an intervention and more accurately identify a cause-and-effect relationship. While other study designs can find causal associations between an intervention and outcome, they cannot rule out that there may be other factors influencing these outcomes. Randomized controlled trials provide greater command over variables, ensuring that the study is measuring what it is meant to be measuring. 

Moreover, randomized controlled trials help to minimize participant and researcher bias. Through the random allocation of participants into the treatment and control groups, the study avoids selection bias, minimizing the influence of demographic information on outcomes. 

Randomized controlled trials that are double-blind further minimize bias, as neither participant or researcher knows who is receiving the intervention. This allows researchers to know if outcomes are due to an intervention or due to the placebo effect, where participants report outcomes because they psychologically feel better, believing they have received treatment. 

While we usually discuss randomized controlled trials in relation to clinical research, they are also valued in other fields. A notable example is the Perry Preschool Project, where American psychologist David Weikart examined how the intervention of high-quality early childhood education would affect the future potential of low-income, barriered children. 

Weikart randomly divided 123 children into either the intervention group or the control group and monitored outcomes from 1962 to 1967, finding that early, high-quality intervention led to positive outcomes in education, economic performance, crime prevention, health, family, and children.¹⁰

Today, randomized controlled trials have transformed medical research, behavioral interventions, and policy-making, helping researchers make evidence-based conclusions about the efficacy of an intervention.

While randomized controlled trials are known as the gold-standard in medical and health research, there are still a few drawbacks. If researchers use simple or block randomization, groups may not have balanced characteristics, skewing the results. 

Additionally, randomized controlled trials require large sample sizes, long durations, and substantial financial resources to conduct effectively. Some interventions may require years to discover significant outcomes or results—and it’s never guaranteed. Additionally, since randomized controlled trials are longitudinal, sometimes taking years, researchers are likely to lose participants along the way.¹¹

The use of a control group that does not receive treatment also comes with some potential issues. If the experiment is not blind, and some participants know they are not receiving the intervention, it is more difficult to get them to continue to share their outcomes. They may lose interest in participating in the study because they do not feel like they are benefitting from it. 

Those that do continue to participate could fall victim to response bias—where people provide inaccurate or false answers to self-report questions—especially if they are part of the treatment group and want to share positive results. One way to combat this is through double-blind experiments, with neither researcher or patient knowing whether an individual is receiving the treatment or placebo. This also negates the impact of the demand characteristic bias, where participants change their behavior to fit the outcome they think the experiment is looking for.

It is also argued that withholding an intervention from a group that would benefit from it—like medication that can help to cure a disease—is unethical. Ideally, all people suffering from the disease would be able to access medication, but to ensure that the medication is effective, randomized controlled trials can be necessary. Being able to more accurately tie an intervention to a positive outcome, as other factors are mitigated through randomization, researchers are more confident in the effectiveness of an intervention, which could help hundreds more down the line. 

Randomized Controlled Trials and Charitable Giving¹²

Charities and non-profit organizations have to compete against one another for donor resources. It is therefore important that these organizations leverage behavioral insights to understand how best to attract and retain donors. 

In 2013, the Zurich Community Trust conducted a study to see how the framing of a donation ask influenced the likelihood that someone would donate. The Zurich Community Trust randomly divided 702 of their existing donors into one of three treatment groups. The first group—the control group—received a message that asked them to make a one-off increase in their donations with the usual options of £1, £2, £3, £5, or £10. Group two received a message that invited them to increase their donations annually with the same amount options as group one. Group three received a message that invited them to increase their donations annually by £2, £4, £6, £8, or £10. 

The Zurich Community Trust found that participants in the control group, who were asked for one-off donations, increased their donations by more than those who were offered the chance to increase donations annually. From this study, the trust could infer that asking donors year over year to increase their donations would result in greater donations rather than asking them from the start to commit to an annual increase. 

Using Randomized Controlled Trials to Test the Effectiveness of Community Therapy¹³

Randomized controlled trials are often used in behavioral science to study the effectiveness of an intervention. Depression and anxiety place a significant burden on health services, however, it can be challenging to convince individuals to seek out professional help. This may be due to the stigma around receiving mental health support, which may be why a community setting could motivate more people to get treatment.

In 2018, researchers in Scotland conducted a study to determine if community-based cognitive behavioral therapy (CBT) had a positive effect on depression and anxiety. Participants were allocated to two groups: one that received treatment immediately, and another that would receive treatment a few months later. The researchers monitored the feelings of anxiety and depression in both groups—comparing the results of the group that had already started attending community therapy groups to those that hadn’t. They found that CBT classes within a community setting were effective for reducing depression and anxiety. 

To make the study stronger and determine that it was the community aspect of therapy that led to positive outcomes, the researchers could have compared a treatment group to another group of participants receiving individual therapy. However, allowing both groups to receive the intervention at different times allowed for more people to receive the help they needed.

How Behavioral Science Informs Policy Making

In this podcast episode, we sat down with Dr. Sarah Ball, a former policymaker, to learn more about how behavioral science and randomized controlled trials influence policymaking. Dr. Ball questions the heavy push and promotion of randomized controlled trials in assessing interventions, which can influence the projects that behavioral scientists work on.

How personalized text messages increased fine repayments by 30%

People often fail to pay their overdue court fines, requiring the need for bailiff interventions that can be timely and costly. In this article, we explore a randomized controlled trial conducted by the Courts Service in the UK, to see if sending personalized text messages to people who owed court fines would push them to pay their fines without needing to send a bailiff. 

1. Morris, T., & Eltoukhi, O. (2024, May 4). How do you choose the best randomization method? LinkedIn. https://www.linkedin.com/advice/0/how-do-you-choose-best-randomization-method
2. Sibbald, B., and Roland M. (January 1998). Understanding controlled trials. Why are randomised controlled trials important? British Medical Journal, 316 (7126), doi: 10.1136/bmj.316.7126.201.
3. Kenton, M. (October 2021) Sample Selection Bias: Definition, Examples, and How to Avoid . Investopedia. Retrieved August 6, 2024, from https://www.investopedia.com/terms/s/sample_selection_basis.asp
4. Milne, I. (2012). Who was James Lind, and what exactly did he achieve? Journal of the Royal Society of Medicine, 105(12), 503–508. https://doi.org/10.1258/jrsm.2012.12k090
5. James Lind Library. (n.d.). Medical Research Council (1948b): The first large-scale controlled trial of a treatment for tuberculosis. Retrieved August 2, 2024, from https://www.jameslindlibrary.org/medical-research-council-1948b/
6. Royal College of Physicians of Edinburgh. (n.d.). James Lind. Retrieved August 2, 2024, from https://www.rcpe.ac.uk/heritage/college-history/james-lind-0.
7. Stavrou, A., Challoumas, D., & Dimitrakakis, G. (2014). Archibald Cochrane (1909–1988): The father of evidence-based medicine. Interactive Cardiovascular and Thoracic Surgery, 18(1), 121–124. https://doi.org/10.1093/icvts/ivt451
8. Elwood, P. (2001). The first, worst, and most successful clinical trial of Archie Cochrane free. Journal of Epidemiology & Community Health, 55(6), 369a. https://doi.org/10.1136/jech.55.6.369a
9. Randal, J. (1999). Austin Bradford Hill: A pioneering force behind clinical trials. JNCI: Journal of the National Cancer Institute, 91(1), 11. https://doi.org/10.1093/jnci/91.1.11
10. HighScope Educational Research Foundation. (2018). Perry Preschool Project: Summary of findings (40th ed.). Retrieved August 3, 2024, from https://highscope.org/wp-content/uploads/2018/11/perry-preschool-summary-40.pdf
11. Simkus, J. (n.d.). Randomized controlled trial. Simply Psychology. Retrieved August 3, 2024, from https://www.simplypsychology.org/randomized-controlled-trial.html
12. Behavioural Insights Team. (2015). Applying behavioral science to charitable giving. https://www.bi.team/wp-content/uploads/2015/07/BIT_Charitable_Giving_Paper-1.pdf
13. Williams, C., McClay, C.-A., Matthews, L., McConnachie, A., Haig, C., Walker, A., & Morrison, J. (2018). A randomised controlled trial of a community based group guided self-help intervention for low mood and stress. British Journal of Psychiatry, 212(2), 88–95. https://doi.org/10.1192/bjp.2017.18