Beyond the Checkbox: Redesigning Surveys for Reliable Behavioral Insights
Surveys are often considered a goldmine for consumer research. They promise insights and predictions about customer behaviors, preferences, and mindsets. But not all surveys are created equal. In fact, some surveys may be steering you in the wrong direction entirely. Why? Because consumers are unique, wacky, creative, illogical, and forgetful. That is, they’re human. Basic surveys often struggle to account for this human-ness, resulting in simplified insights that don’t represent the end user.
So how can you ensure that your research is providing you with valuable insights and not fool’s gold? And what “bias-reduced” methods can you leverage to reliably understand and predict consumer behaviors? Today we’ll unravel some of the biases present in traditional surveys and introduce three methods that can offer deeper insights into the baffling nature of human behavior.
The “human-ness” of answering simple questions
Think back to your last doctor’s appointment and the ever-troubling questions of: “How much do you exercise? Do you drink enough water? How many hours of sleep do you get a night?” If you’re a fitness buff, water enthusiast, and night-time guru, then you may breeze past these questions without a second thought. But if you’re anything like me (and countless other regular folks), you may be inclined to lie, just a little. Maybe you’d say you sleep eight hours instead of your actual six. Or, you’d say you exercise every day for 30 minutes instead of your actual 10-minute walk to and from the subway.
The comforting (yet perhaps troubling) reality is that you’re not alone in wanting to stretch the truth. The response bias is a commonly known phenomenon not just in the doctor’s office, but also when conducting surveys. Our tendency to respond to questions in a way deemed “socially acceptable” leads us to exaggerate positive behaviors and minimize negative ones.
Wanting to “look good” in surveys is not the only human thing we do that impacts potential responses. There are countless other cognitive biases that sway consumers to report behaviors different from those they actually practice. They may experience choice overload when deciding between multiple options, encounter the optimism bias when predicting their future behavior, or face decision fatigue when answering an onslaught of survey questions.
What this means for survey-based research
So what does this mean for our beloved surveys? Should we unsubscribe from our favorite survey platforms or toss our research out the window? Of course not! We just need to be conscious about how we design our research so that we can minimize biases and maximize our understanding of consumers’ true behavior. Specifically, we need to consider ways that we can assess behavior beyond classic multiple-choice or select-all driven surveys.
Our favorite bias-reduced survey methods
We’re going to explore three research methods that maximize our ability to understand actual behavior. Most importantly, these methods can all be easily deployed through an online survey platform! Although real observation is ideal, we know it’s often impractical – whether you’re constrained by time, geographic location, budget, or language.
There are, of course, many different ways to enhance survey design. These are just some of our favorite “bias-reduced” methods here at TDL! By using best-worst scaling, tasks and scenarios, and game-based design, we aim to reduce common biases to tap into insights that are reflective of real-world behaviors.
Best-worst scaling
In best-worst scaling, respondents compare different options and pick the top and bottom of the pile. (Think of this as sorting favorites from the not-so-favorites.) Options are then repeatedly randomized and shown in different combinations. It’s a pretty straightforward way to quickly reveal consumer priorities.
Superpower: Provides a prioritized list of options based on consumer preferences and aversions.
Overcoming Bias: Reduces choice overload by simplifying complex preferences to simple options.
Survey Upgrade: Transform ranking questions into a best-worst analysis. After all, who among us can really rank 6+ different options?
TDL in Action: Best-worst scaling is one of our go-tos when weighing options for a new behavioral intervention, or ranking features for a website or mobile application.
Tasks and scenarios
Why would you ask a participant what they would do when you could actually get them to do it? For this approach, participants complete pre-set tasks while you capture information like click distributions or selected buttons. For example, participants can navigate to specific pages on your website, find the answer to a question, or interact with an application, all while you measure their behavior!
While traditionally done for interactions with products or digital services, this method can also be used more creatively by having your participants imagine themselves in a scenario before answering a series of traditional survey questions.
Superpower: Assesses firsthand how people would behave in a given scenario – no guesswork involved.
Overcoming Bias: Sidesteps the optimism bias. It's about actions, not just well-meaning intentions.
Survey Upgrade: Replace questions about product interaction (e.g., “How easy is it to find the red button”) with specific tasks (e.g., Find the red button) and metrics (e.g., Time to find the red button).
TDL in Action: Task-based questions are heavy hitters when analyzing navigation behavior on digital interfaces.
Game-Based Designs
Game-based designs transform traditional survey questions into interactive experiences. Participants might progress through levels, earn points, or face challenges that are directly linked to overall research questions. Want to know how someone would spend their money on the stock market? Or how they would design their ideal workout routine? Gamify it!
Superpower: Captures in-the-moment experiences and behaviors.
Overcoming Bias: Eliminates decision fatigue by creating an engaging experience. Let’s be real – we’d all rather do a “which animal are you?” quiz instead of answering redundant questions on a basic survey design.
Survey Upgrade: Create simple games or quizzes as substitutes for hypothetical questions about preferences or behaviors.
TDL in Action: Game-based designs are a known and loved tool for some of our finance-related experiments!
Revamping your research
While surveys have their limitations, they’re not inherently flawed. By leveraging principles of behavioral science to select research methods, we can develop surveys that more accurately reflect real, human behavior.
As you delve into your next round of research, consider: how easy is it for someone to stretch (or forget!) the truth while answering your survey? Are you asking your consumer to predict their future behavior? If so, what bias-reduced methods could work for you?
And of course, drop us a line! Do any of these methods speak to you? Have you tried any of them and had success? Are you looking for support to implement these (or other) bias-reduced research methods? We’d love to hear from you!
About the Author
Alexi Michael
Alexi is a Consultant at The Decision Lab. Her expertise is multidisciplinary, spanning the fields of social innovation, artificial intelligence, and human-centered design.
About us
We are the leading applied research & innovation consultancy
Our insights are leveraged by the most ambitious organizations
“
I was blown away with their application and translation of behavioral science into practice. They took a very complex ecosystem and created a series of interventions using an innovative mix of the latest research and creative client co-creation. I was so impressed at the final product they created, which was hugely comprehensive despite the large scope of the client being of the world's most far-reaching and best known consumer brands. I'm excited to see what we can create together in the future.
Heather McKee
BEHAVIORAL SCIENTIST
GLOBAL COFFEEHOUSE CHAIN PROJECT
OUR CLIENT SUCCESS
$0M
Annual Revenue Increase
By launching a behavioral science practice at the core of the organization, we helped one of the largest insurers in North America realize $30M increase in annual revenue.
0%
Increase in Monthly Users
By redesigning North America's first national digital platform for mental health, we achieved a 52% lift in monthly users and an 83% improvement on clinical assessment.
0%
Reduction In Design Time
By designing a new process and getting buy-in from the C-Suite team, we helped one of the largest smartphone manufacturers in the world reduce software design time by 75%.
0%
Reduction in Client Drop-Off
By implementing targeted nudges based on proactive interventions, we reduced drop-off rates for 450,000 clients belonging to USA's oldest debt consolidation organizations by 46%
