Bridging The Divide Between Decision Science And Policy

by Craig Fox & Sim Sitkin

There are many stories of behavioral scientists who are resourceful, entrepreneurial, determined, and idealistic can successfully push their ideas into policy and practice. However, the vast majority of rank-and-file scientists lack the resources, time, access, and incentives to directly influence policy decisions. Meanwhile, policymakers and practitioners are increasingly receptive to behavioral solutions but may not know how to discriminate good from bad behavioral science. A better way of bridging this divide between behavioral scientists and policymakers is urgently needed. The solution, we argue, requires behavioral scientists to rethink the way they approach policy applications of their work, and it requires a new vehicle for communicating their insights.

Rethinking the approach to decision science research

Behavioral scientists interested in having real-world impact typically begin by reflecting on consistent empirical findings across studies in their research area and then trying to generate relevant applications based on a superficial understanding of relevant policy areas. We assert that to have greater impact on policymakers and other practitioners, behavioral scientists must work harder to first learn what it is that practitioners need to know. This requires effort by behavioral scientists to study the relevant policy context—the institutional and resource constraints, key stakeholders, results of past policy initiatives, and so forth—before applying behavioral insights. In short, behavioral scientists will need to adopt a more problem-driven approach rather than merely searching for applications of their favorite theories.

This point was driven home to us by a story from David Schkade, a professor at the University of California, San Diego. In 2004, Schkade was named to a National Academy of Sciences panel that was tasked with helping to increase organ donation rates. Schkade thought immediately of aforementioned research showing the powerful effect of defaults on organ donation consent. Thus, he saw an obvious solution to organ shortages: Switch from a regime in which donors must opt in (for example, by affirmatively indicating their preference to donate on their driver license) to one that requires people to either opt out (presume consent unless one explicitly objects) or at least make a more neutral forced choice (in which citizens must actively choose whether or not to be a donor to receive a driver’s license).

As the panel deliberated, Schkade was surprised to learn that some states had already tried changing the choice regime, without success. For instance, in 2000, Virginia passed a law requiring that people applying for driver’s licenses or identification cards indicate whether they were willing to be organ donors, using a system in which all individuals were asked to respond (the form also included an undecided category; this and a nonresponse were recorded as unwillingness to donate). The attempt backfired because of the unexpectedly high percentage of people who did not respond yes.1,2

As the expert panel discussed the issue further, Schkade learned that a much larger problem in organ donation was yield management. In 2004, approximately 13,000–14,000 Americans died each year in a manner that made them medically eligible to become donors. Fifty-nine different organ procurement organizations (OPOs) across the United States had conversion rates (percentage of medically eligible individuals who became donors in their service area) ranging from 34% to 78%.1 The panel quickly realized that getting lower performing OPOs to adopt the best practices of the higher performing OPOs—getting them to, say, an average 75% conversion rate—would substantially address transplant needs for all major organs other than kidneys. Several factors were identified as contributing to variations in conversion rates: differences in how doctors and nurses approach families of potential donors about donation (family wishes are usually honored); timely communication and coordination between the hospitals where the potential donors are treated, the OPOs, and the transplant centers; the degree of testing of the donors before organs are accepted for transplant; and the speed with which transplant surgeons and their patients decide to accept an offered organ. Such factors, it turned out, provided better opportunities for increasing the number of transplanted organs each year. Because almost all of the identified factors involve behavioral issues, they provided new opportunities for behavioral interventions. Indeed, since the publication of the resulting National Academy of Sciences report, the average OPO conversion rate increased from 57% in 2004 to 73% in 2012.3

Extrapolating decision science research to the real world

The main lesson here is that one cannot assume that even rigorously tested behavioral scientific results will work as well outside of the laboratory or in new contexts. Hidden factors in the new applied context may blunt or reverse the effects of even the most robust behavioral patterns that have been found in other contexts (in the Virginia case, perhaps the uniquely emotional and moral nature of organ donation decisions made the forced choice regime seem coercive). Thus, behavioral science applications urgently require proofs of concept through new field tests where possible. Moreover, institutional constraints and contextual factors may render a particular behavioral insight less practical or less important than previously supposed, but they may also suggest new opportunities for application of behavioral insights.

A second important reason for field tests is to calibrate scientific insights to the domain of application. For instance, Sheena Iyengar and Mark Lepper famously documented choice overload, in which too many options can be debilitating. In their study, they found that customers of an upscale grocery store were much more likely to taste a sample of jam when a display table had 24 varieties available for sampling than when it had six varieties, but the customers were nevertheless much less likely to actually make a purchase from the 24-jam set.4 Although findings such as this suggest that providing consumers with too many options can be counterproductive, increasing the number of options generally will provide consumers with a more attractive best option. The ideal number of options undoubtedly varies from context to context,5 and prior research does not yet make predictions precise enough to be useful to policymakers. Field tests can therefore help behavioral scientists establish more specific recommendations that will likely have greater traction with policymakers.

Effectively communicating decision science insights

Although a vast reservoir of useful behavioral science waits to be repurposed for specific applications, the kind of research required to accomplish this goal is typically not valued by high-profile academic journals. Most behavioral scientists working in universities and research institutes are under pressure to publish in top disciplinary journals that tend to require significant theoretical or methodological advances, often requiring authors to provide ample evidence of underlying causes of behavior. Many of these publications do not reward field research of naturally occurring behavior,5 encourage no more than a perfunctory focus on practical implications of research, and usually serve a single behavioral discipline. There is therefore an urgent need for new high-profile outlets that publish thoughtful and rigorous applications of a wide range of behavioral sciences—and especially field tests of behavioral principles—to increase the supply of behavioral insights that are ready to be acted on.

On the demand side, although policymakers increasingly are open to rigorous and actionable behavioral insights, they do not see much research in a form that they can use. Traditional scientific journals that publish policy-relevant work tend to be written for experts, with all the technical details, jargon, and lengthy descriptions that experts expect but busy policymakers and practitioners cannot decipher easily. In addition, this work often comes across as naive to people creating and administering policy. Thus, new publications are needed that not only guarantee the disciplinary and methodological rigor of research but also deliver reality checks for scientists by incorporating policy professionals into the review process. Moreover, articles should be written in a clear and compelling way that is accessible to nonexpert readers. Only then will a large number of practitioners be interested in applying this work.

References:

1. Committee on Increasing Rates of Organ Donation, Childress, J. F., & Liverman, C. T. (Eds.). (2006). Organ donation: Opportunities for action. New York, NY: National Academies Press.

2. August, J. G. (2013). Modern models of organ donation: Challenging increases of federal power to save lives. Hastings Constitutional Law Quarterly, 40, 339–422.

3. U.S. Department of Health and Human Services. (2014). OPTN/SRTR 2012 Annual Data Report. Retrieved from http://srtr.transplant.hrsa.gov/annual_reports/2012/Default.aspx

4. Iyengar, S. S., & Lepper, M. R. (2000). When choice is demotivating: Can one desire too much of a good thing? Journal of Personality and Social Psychology, 79, 995–1006.

5. Shah, A. M., & Wolford, G. (2007). Buying behavior as a function of parametric variation of number of choices. Psychological Science, 18, 369–370.

6. Cialdini, R. B. (2009). We have to break up. Perspectives on Psychological Science, 4, 5–6.

This article originally appeared in [https://behavioralpolicy.org/article/bridging-the-divide/] and belongs to the creators.