Rethinking Voter Preferences: A New Approach to Understanding Election Day Choices

Let’s start by summarizing some key limitations before we get into the good stuff. Some of these limitations are specific to this particular experiment such as limited sample size, while others are inherent to the methodology itself. In both cases, but particularly the latter, we advise caution in interpreting these results as being representative of the greater voting population. This should be taken only as a proof of concept.

Limitations specific to this experiment:

• I’m not a political scientist: I’m a nerd who finds alternative methods of understanding sentiment and preference interesting. If you’re another kind of nerd who considers themselves an expert on politics and policy or represents an organization composed of such nerds, reach out. I’d love to collaborate on an improved version of this!
• I’m liberal: Our dataset, while small, is evenly split between Democrats and Republicans, but I myself, for better or for worse, am a liberal. I promise to try to keep my own personal biases out of the interpretation that follows, but I doubt that I will be perfectly successful. The Decision Lab is a nonpartisan organization and I should stress that any views that shine through are my own.
• Our sample is small: These results reflect an experiment I conducted with a sample of 100 voters. In general, discrete choice experiments can function well with smaller samples, and we did detect a number of statistically significant results. However, a sample of 100 is simply unlikely to be truly representative of the broader population. As such, we do not specify which estimates are statistically significant and all graphs carry clear disclaimers that all estimates—even those that may be statistically significant—should be considered non-representative. 
• Our choice sets are small: Given our small sample, we only explore five candidate attributes. These are age, stance on Israel/Gaza, women’s reproductive rights, immigration, and the peaceful transfer of power after the 2024 election. These were selected in part due to their relevance in the broader political discourse, major poll results, and an available and reasonably articulated stance from each of the two primary 2024 presidential candidates. A larger sample would permit the inclusion of a more expansive set of issues.
• This data was collected in early March of 2024, and thus stances ascribed to individual candidates may no longer be accurate. 

If you’re interested in partnering with us to conduct an expanded experiment with a larger sample and more issues, submit a collaboration request here!

Here were the methodological limitations:

• Number of issues: It is realistic to expect an individual to be able to consider and respond to a series of 30 questions on the importance of 30 different issues. However, making a choice between two options, each with 30 dimensions, is unrealistic. Such a complex decision encourages overreliance on heuristics and may challenge interpretation. While there is no maximum number of attributes, we tried to limit our experiment to between 8-12 attributes, depending on the complexity of the options within them.
• Articulating Positions: While this limitation is not unique to DCEs, identifying a political candidate’s position on a given issue may be challenging in the event that no stance has been included on a formal platform.

For the nerds out there, Jens Hainmueller’s 2015 paper “Causal Inference in Conjoint Analysis: Understanding Multidimensional Choices via Stated Preference Experiments” provides a thorough exploration of the applicability, strengths, and limitations of DCEs and other forms of conjoint analysis for research in the social sciences.

First things first, let’s take a look at the experiment from the participant’s perspective. You’ll notice that Candidates A and B are different in most, but not in every regard. While this stems from the randomized way the choice sets are constructed, it also offers a valuable opportunity to better understand single-issue voters. Consider a hypothetical voter who really cares about the southern border. By comparing two candidates with the same stance on the southern border, you are able to build a more nuanced picture of their preferences.

Now that the experience is a bit easier to picture, let’s map out the options that were included in this particular experiment. Due to our modest sample, we included five categories in total. Each category, or issue, had 2-4 possible states. One option aligned with Joe Biden, one with Donald Trump, with up to two alternatives. These alternatives may be arbitrary (in the case of age), a continuation of the current legislative status quo (in the case of abortion and the southern border), or an espoused view of some other political figure or social movement (in the case of conditioning aid to Israel).

When the data from a DCE is analyzed, each element contained within, such as the age of 65 years, is assigned something called a marginal component effect. This number can be thought of as a score or “weight” that represents the extent to which each item impacted the choices made. 

Let’s get a better idea of how this works by taking a look at how the Democrats and Republicans in our experiment were influenced by the three policy positions on the Southern Border.

You’ve probably already noticed something important: each of the two graphs below has one issue with a score of “0.” This is a necessary detail of the analytical method used to compute these weights. Within every category, there will be one reference element assigned a score of 0. For the Southern Border, our baseline issue is to stick with the status quo. We see that Democrats would rather things stay the same than go with Trump’s proposed direction, but would much rather move towards a slate of policies that resemble the contents of February’s bipartisan National Security Agreement. That bill was broadly viewed as a concession by liberals in light of the perceived criticality of securing aid to Ukraine. Despite it not being all that popular amongst progressives, this serves as a harsh indictment of the status quo along with Trump’s proposed direction. 

We see a different picture emerge when we look at Republican voters. Within our cohort of respondents, we see that Trump’s proposed direction—emphasizing a focus on domestic deportations and a reallocation of personnel to the southern border—jumps from last place to first. We do, however, observe that the big gulf does not lie between Trump’s prognostication and Biden’s policy endorsements, but rather between either of those options and staying our current course. Political junkies may see this as an affirmation that pressure from Trump to shut down February’s immigration bill was an effort to deprive the Biden campaign of a political victory by avoiding an outcome that could have been broadly popular amongst Republicans.

Of course, that analysis has been floating around for a while. The magic of the discrete choice experiment is not reaffirming existing beliefs about the motivations of political actors, but providing a lens through which all of these trade-offs can be viewed on a single scale.

All of the options included in the experiment are analyzed in a single model, thus all of the calculated weights exist on a single scale. For the nerds out there, this number represents the change in log odds relative to choosing one alternative over another. If that sounds like gibberish, rest easy by knowing that bigger positive numbers mean that a choice set containing that element is more likely to be chosen, while bigger negative numbers mean a choice set containing that number is less likely to be chosen. By looking at the gap between the best and worst-performing options within a category, we can get a rough sense of the ‘importance’ of that category. Importance, in this case, can be thought of as the size of the penalty for possessing the worst lowest weighted element in a category relative to the most valued element. If that disparity is large, then that particular category will exert a large impact on decisions voters make.

With this in mind, let’s look at issue importance within our experiment. The below set of graphs shows overall importance in each category for both Republicans and Democrats. As this chart contains all positions, not just those attributed to Biden or Trump, it’s important to remember that this shouldn’t be construed as the strength of either candidate. These figures represent the ‘available space’ within each category. Whether you’re 45, 65, 77, or 81 will, in general, matter less to Democrats than whether or not you’ve committed to a peaceful transition of power after the 2024 election. In contrast, we see that there’s actually quite a bit of distance in the Age question for Republicans, while there is comparatively less between the included issues for Israel/Gaza or the transfer of power. 

This second set of graphs focuses on the positions assigned to either Joe Biden or Donald Trump. You’ll notice a few differences. A fair few of these are to be expected from some of the trends we saw earlier.

Democrats do slightly prefer a 77-year-old candidate to an 81-year-old candidate, however exert a strong preference for Biden in virtually every policy domain included. This contrast is most extreme for the difference in commitment to a peaceful transfer of power, along with their stances on reproductive rights. Positions on Israel/Gaza and the Southern Border are viewed comparatively favorably to Trump, and it’s probably worth noting that these are two issues in which the party is thought to be out of step with the people that compose it.

For political news junkies, perhaps the biggest shock might be that, when we look at Republicans, Trump’s abortion stance is perceived fairly positively. It’s important to recognize that Trump’s 16-week ban (which reports indicates has shifted to 15 weeks since this data was collected) is less restrictive than a six-week ban, and our language did not clarify a position on exceptions for rape or incest, both of which could be poised to add fuel to the fire of unpopularity, though Trump appears likely to support such exceptions. 

This, to an extent, contradicts a commonly held view on the left that reproductive rights are a universal area of strength for Democrats, along with perceptions on the right that it would be politically wise to avoid national legislation and leave the choice to the states. This is where we once again stress that this data is from a small sample, insufficiently powered to delve deeper into differences that exist between demographic subgroups of the party, notably women.

The last item on our list is to demonstrate how this method can map out opportunities for a political candidate by identifying chances to adopt new positions that might improve their political chances. Now, if this is your objective as a strategist, your discrete choice experiment should include more positions not currently held by your candidate of choice, but that might be politically viable paths forward. Keep that in mind, and interpret this as an illustration of possibility, rather than a recommendation, as our alternative stance for most issues was to simply continue with the status quo.

When we look at Biden, his best bet would be to suddenly become about 16 years younger. If that fails, then in the context of our experiment, his position on Israel and Gaza presents the largest opportunity to gain traction among the Democrats included, with the top performing item in that category being to condition financial and military support on Israel abstaining from violating the human rights of civilians, ceasing indiscriminate bombings, and committing to participating in talks towards a two-state solution. 

While Trump shouldn’t turn down anyone who comes his way with a magical de-aging potion, his biggest opportunity, amongst those issues included in our experiment, also relates to Israel and Gaza. His stated desire to continue financial and military support to Israel compounded with ideological screenings on refugees and the revoking of student and visitor visas for Palestinians is in fact the worst performing option amongst Republicans, while the stance that aligns with the Biden administration’s status quo is preferred. He also, and this may be surprising for some and unsurprising for others, would stand to gain amongst Republicans by committing to honor the results of the 2024 election. 

As you probably guessed, ample insight can be drawn from this method of assessing preference. All told, we looked at raw preferences within one out of five issues, the overall importance of the issue itself, a comparison of how candidates fare against one another, and an assessment of where opportunity may lie for each candidate within their party.

Now, you probably noticed that the structure of this discrete choice experiment puts limits on the quality of the insights we can (and should) draw from this proof of concept. Beyond the obvious limitation around sampling, a truly useful DCE for, say, identifying opportunities for candidates to evolve their policy stances in politically beneficial directions, would want to include more stances that reflect directions a candidate could take, to effectively contrast them with where they’re starting. A version tuned for detecting the direction in which the current political winds are blowing would focus on including a broader set of issues through which to compare the two leading candidates. A more thoughtful version of this exercise might go beyond superficials like “age” and towards the pieces of the puzzle that make somebody seem “old,” such as stumbling over their words or feet, visible gaffes, moments of irritation and anger, or confusing the names of loved ones, world leaders, or political opponents. 

You should not leave this article believing that you better understand the wims of the American voting public—you don’t, and I hope I’ve made that abundantly clear throughout. What I hope is that you leave this article curious about alternative ways of exploring what drives the decisions made on voting day, and perhaps an interest in this method in particular. If so, so are we, so give us a shout. We’d love to explore that possibility together.