In one episode of the NBC sitcom Parks and Recreation, our protagonist, Leslie Knope, becomes embroiled in a political scandal when her efforts to secure funding for her governmental department result in the imminent closure of a local animal shelter, imperiling its resident cats and dogs. Wasting no time, one of Leslie’s political opponents (played by the delightful Kathryn Hahn) appears on local TV for comment.
“I’m not saying Leslie Knope is a dog murderer, per se,” Hahn’s character says smilingly. “I just think her actions raise some questions. Like, for example, is she a dog murderer?”
“Well, I don’t know the answer to that, Jennifer,” says the show’s host gravely, “But your tone makes me think, yes.”
This absurd scene is, of course, fictional. But in the real world, you may have had the displeasure of overhearing some similarly cynical questions: “Do vaccines really work?” “Aren’t there any alternative treatments?” “Shouldn’t we just let COVID do its thing?”
Sometimes, questions like these are sincere, and our job as behavioral scientists is to answer and navigate them in earnest. Other times, however, they are used to sow misinformation and distrust among the general public. In these cases, our job as behavioral scientists is instead to figure out how to stop bad questions from inferring with good policymaking.
To do that, though, we need to figure out why questions can be misleading in the first place. Thankfully, there is already quite a bit of work, in psychology, linguistics, and philosophy, that can help us understand why questions can lead us astray.
Behavioral Science, Democratized
We make 35,000 decisions each day, often in environments that aren’t conducive to making sound choices.
At TDL, we work with organizations in the public and private sectors—from new startups, to governments, to established players like the Gates Foundation—to debias decision-making and create better outcomes for everyone.
How do bad questions work? A psychological perspective
Questions, just like statements, can trigger all sorts of cognitive biases. For example, consider framing effects, as first proposed by Tversky and Kahneman 19811 (and recently replicated in 2015).2 Framing effects occur when individuals respond differently to the same information: for instance, people respond more positively to “the glass is 50% full” than to “the glass is 50% empty,” even though the statements convey the exact same information.
However, many studies point to similar effects occurring in public opinion surveys. Two questions might solicit the same information, but, based on how the question is framed, participants might give entirely different answers.3
This is an intuitive point. Compare these two questions, from a 2003 Pew Poll:4
- Do you favor or oppose taking military action in Iraq to end Saddam Hussein’s rule?
- Do you favor or oppose taking military action in Iraq to end Saddam Hussein’s rule even if it meant that U.S. forces might suffer thousands of casualties?
For the first question, 68% of participants said they favored military action, while 25% said they did not.4 But when the cost of going to war was made explicit in the second question, their attitudes changed: only 43% said they favored military action, whereas 48% did not.4 Even though the questions ask for the same information, the framing effect led to a massive shift in how participants responded.
A linguistic perspective
Framing effects come into play with all kinds of utterances, statements included. Questions, however, have their own, unique linguistic properties. According to most standard models, questions act as a collection of possibilities.5 Additionally, questions are then added to what some linguists call “the Question Set”: the questions everyone in a conversation is committed to answering.6,7 When I ask “Where is my ice cream?” I am asking others to sort through the set of possibilities to fill in the blank—“His ice cream is at ___”—so that we can figure out what the right answer is.
A good question gives us the right blank to fill in. It chops up the possibility space in a way that lets us sort through the options that will let us accomplish our goals (e.g., finding the truth). A bad question, in contrast, chops things wrong. It forces us to spend time sorting through the wrong possibilities, distracting us from the issues we really want to solve.
This is a common, but subtle, occurrence. For instance, some people within institutions of higher learning ask “Will diversity harm our research?”, prompting others to spend time trying to assure them that it won’t. However, in spending time sorting through the possibilities of “diversity harms rigor” and “diversity does not harm rigor,” we end up forgetting about the possibility that diversity could help research.8,9 (In fact, the historical record of figures like Francis Cecil Sumner, Albert Sidney Beckham, and Kenneth and Mamie Clark indicate that this is almost certainly the case.) Even without any malice, the question derails the conversation from the possibilities that really matter.
A philosophical perspective
It’s worth remembering, though, that communication does not exist in a vacuum. We don’t just know our native language(s); we use them. And just like any other human action, what we do with our words is governed by certain sets of norms.
Many epistemologists—people who study the nature of knowledge—have recently written about the norms that govern how we communicate what we know. Famously, the philosopher Timothy Williamson argues for a knowledge norm of assertion: don’t assert what you don’t know.10
Importantly, we assume that others follow this norm, too. This is why, when someone says “you left your ice cream at home,” I am inclined to think they’re right. I assume they know what they’re saying because they’ve said it.
Recently, this approach has been extended to questions, too. And while we don’t yet know what the right norm is (that’s philosophy for you!), one thing we do know is that, when someone asks a question, we assume they don’t know the answer.11 This is fairly intuitive: it’d be really weird for me to keep asking you where my ice cream is if I secretly knew it’s been in my freezer all along.
In a healthy, cooperative conversation, this norm serves us well. It allows us to ask questions to solicit information we need, and it spares our conversational partner from having to explain what we already know.
Malicious actors, however, can use this norm to their advantage. They can use questions to feign ignorance and pretend that they are just engaging in healthy inquiry. This tactic even has a name: “I’m just asking questions!” A bad actor will ask a clearly inflammatory question; but when met with backlash, they will simply say that they are innocently inquiring because they don’t know the right answer. They exploit the fact that questions signal ignorance in order to hide their ulterior motives.
A Case Study
Let us return to the questions we started with:
- Do vaccines really work?
- Aren’t there any alternative treatments?
- Shouldn’t we just let COVID do its thing?
For starters, notice that all three of these questions frame things in a negative light. They paint the picture that our current means to reduce the spread of COVID are not working, and the last one even frames deaths from COVID as an inevitable fact of life. This, of course, ignores the fact that 1) our current measures (masks, vaccines, and social distancing) are incredibly effective, and that 2) our goal should be to reduce deaths wherever it’s feasible to do so, especially in a pandemic.
Additionally, these questions chop up the possibility space in a way that leads us astray. We need to address important questions surrounding, say, how to deliver vaccines to groups that have borne the worst brunt of the pandemic (and who have been traditionally underserved by the medical system), or how to establish post-pandemic accommodations for the people who need them. Instead, the questions above obscure these important issues, by having us sort through a possibility space we have already explored.
Finally, and perhaps most importantly, these questions are often not asked in earnest. Many figures use these questions to delegitimize otherwise effective policy interventions, or, more broadly, to promote their political or financial interests. When confronted, however, they simply resort to saying that they are just “asking questions” because they want to find out the truth. (They usually lament how “people can’t just ask questions anymore.”) In reality, they have usually made up their mind. They just do not want to say the quiet part out loud.
In theory, questions just help us solicit information we don’t know. In reality, though, what we ask has an impact on how a conversation goes about. Good questions lead us down the right path, towards the truth or some other goal we have. Bad questions lead us astray, and malicious actors can weaponize them to shape a conversation in a way that only benefits them.
Sometimes, as Kathryn Hahn’s Parks and Rec character demonstrates, a question is just meant to get us to say “yes.” But as our media ecosystem becomes increasingly polarized—and as more people receive their news from outside mainstream media12—we have to be vigilant of the rhetorical effects questions can have. Otherwise, we’ll be unable to see how a bad question steers public discourse away from where it ought to go, and we’ll be unable to get it back on track.
Remember: just because someone is—even sincerely— “just asking questions” does not mean those questions deserve answers.
- Tversky, A., & Kahneman, D. (1981). The framing of decisions and the psychology of choice. Science, 211(4481), 453–458. https://doi.org/10.1126/science.7455683.
- Owens, B. (2018). Replication failures in psychology not due to differences in study populations. Nature, d41586-018-07474-y. https://doi.org/10.1038/d41586-018-07474-y
- Keren, G. (Ed.). (2011). Perspectives on framing. Psychology Press.
- Pew Research Center. (n.d.). Writing Survey Questions. https://www.pewresearch.org/our-methods/u-s-surveys/writing-survey-questions/
- Cross, Charles and Floris Roelofsen, “Questions”, The Stanford Encyclopedia of Philosophy (Fall 2020 Edition), Edward N. Zalta (ed.), URL = <https://plato.stanford.edu/archives/fall2020/entries/questions/>
- Fleisher, N. (2021, October 15). Can He Do That? Thinking C21. https://www.c21uwm.com/2020/10/15/can-he-do-that/
- Portner, P. (2004). The Semantics of Imperatives within a Theory of Clause Types. Semantics and Linguistic Theory, 14, 235. https://doi.org/10.3765/salt.v14i0.2907
- Robertson, E. (2013). The Epistemic Value of Diversity: The Epistemic Value of Diversity. Journal of Philosophy of Education, 47(2), 299–310. https://doi.org/10.1111/1467-9752.12026
- Eswarman, V. (2019, April 29). The Buisness Case for Diversity in the Workplace Is Now Overwhelming. World Economic Forum. https://www.weforum.org/agenda/2019/04/business-case-for-diversity-in-the-workplace/#:~:text=There%20is%20substantial%20research%20to,and%20better%20problem%2Dsolving%20abilities
- Williamson, T. (1996). Knowing and Asserting. The Philosophical Review, 105(4), 489. https://doi.org/10.2307/2998423
- Whitcomb, D. (2017). One Kind of Asking. The Philosophical Quarterly, 67(266), 148–168. https://doi.org/10.1093/pq/pqw027
- Shearer, E., & Mitchell, A. (2021, January 12). News use across social media platforms in 2020. Pew Research Center’s Journalism Project. https://www.pewresearch.org/journalism/2021/01/12/news-use-across-social-media-platforms-in-2020/
About the Author
Juan Ignacio Murillo
Juan is a Summer Associate at The Decision Lab. He recently graduated from the University of Toronto with a Bachelor’s degree in philosophy and linguistics, and is currently pursuing an MA in Philosophy at the University of Wisconsin-Milwaukee. He is passionate about integrating and applying traditional philosophical thinking—especially in metaethics, the philosophy of language, and the philosophy of science—to empirical research and problems in everyday life. Currently, he is interested in what values are, and how they feature in what we say and how we think. He is also interested in how understanding the role values play in our lives may help us deal with broader societal issues, such as vaccine hesitancy.