Game, Set, Match: Getting Gamification Right

Funny story: I started using Duolingo, the language-learning app, exactly 427 days ago. My niece had begun French classes and I wanted to learn French, so we could converse without others intercepting us. She stopped her classes at school within 3 months. Meanwhile, I am stuck because I now have a 427-day streak on Duolingo and I cannot give that up. Actually, that’s not true; I can give it up, I just don’t want to.

At the other end of the room, my partner is busy playing FIFA 2021 on his PlayStation. I hear loud celebratory music. He opens a reward box. The screen becomes dark and then, amidst applause, out walks the latest player being added to his team. The drama of all of it is jarring and yet, addictive. I find myself waiting for him to open his weekly rewards.

When I question why he wastes time playing the game, he counters me by asking why that streak on Duolingo matters to me. I argue it’s different: I am learning a new language, thanks to that streak. But, deep down, I know he is right. We are both addicted to a form of engagement. His, a video game. Mine, gamification.

What is gamification?

On any normal workday as an applied behavioral scientist, I would describe gamification as one of those annoying, trite buzzwords people throw around in meetings to appear cool and up-to-date with technological trends. It stands right up there, alongside “digital transformation,” “AI,” “machine learning,” and “the Metaverse.” 

When someone mentions this G-word in response to a question on customer engagement, my conditioned response is to cringe. Why? Because when boardrooms talk about gamification, it is almost always as a high-level notion of some form of customer engagement, involving standard tools such as badges and leaderboards. In truth, gamification is a lot more nuanced than that—or, at least, it should be.

But today, I’m not speaking as a practitioner. Here, I am going to discuss gamification from a first-principles perspective, to explain why it is important and why it has to be done right. A “first-principles” approach would entail a more fundamental understanding of the underlying principles behind games and then applying them to a problem, rather than viewing gamification as a quick-fix solution.

Why do we play games? 

Have you ever played Candy Crush? The colors, the drama, the sounds; getting sudden boosters, seeing yourself rise up the ranks on a leaderboard, progressing to harder levels—and then being told you have to wait 15 minutes to play the next game. Many psychological effects are at work here:

  1. Scarcity: Having limited attempts available makes the game feel more valuable than it really is. (So valuable that many people fork over their cash to regain access immediately.)
  2. Social Norms: It’s not just about performing well in the game, it’s about getting your friends to see that you are doing better than them.
  3. Unpredictability: What booster you will get, and when, is a surprise. This element of randomness has long been known by psychologists to be more habit-forming than predictable rewards.
candy crush saga lives
Source: Business Insider1

Consider something more complex—for example, popular games like Fortnite or Animal Crossing. You don a new avatar and become a new person; you work towards goals like achieving new personal bests or upgrading your house; you encounter random rewards and form relationships with other players (whether real people or cute computerized animals). You can once again see many psychological effects at play here: self-image, intrinsic motivation, unpredictability, social norms, and so on.

These games function very differently, but they have a lot in common. They are both extremely engaging: even children can learn their rules and get absorbed in the gameplay. And they both rely heavily on user psychology and behavioral science to keep players interested. Game designers have an acute understanding of what gets users to engage, what actions feel intuitive, and what drives players to keep playing. They build this into the various mechanisms they use to make games fun.

From games to gamification

What, then, is gamification? I would say, from a first-principles perspective, gamification should be about bringing in the game designer’s mindset into designing any interface, be it a health app, an eCommerce app, a dating app, or a social network. 

But a more formal definition available on the internet would be “adding game-like elements to a non-game environment.” I personally believe this distinction between the first-principles definition and the formal definition is very important. When we describe gamification as “adding game-like elements,” we are automatically boxing ourselves into a process where we identify what these game-like elements are and then force them to fit any non-game context, such as a website.

In most companies, the game mechanics people think of first are things like badges, leaderboards, and scratch cards. There are several frameworks, some very popular, that recommend this approach. For example, Octalysis identifies 8 principles and a few hundred techniques of gamification.2 They have, through years of playing games, identified “mechanisms” and offered them as a toolbox to interested people. 

Here’s the problem: We then go on to force-fit these identified mechanisms into our own contexts, even when they’re not necessarily a good fit. (As the popularity of gamification has skyrocketed, this has led many companies to have awkward discussions like this.)

Don’t get me wrong: identifying basic mechanisms of gamification is a great starting point. My discomfort comes from the second part: people forcing these mechanisms into any and every context. 

We often fail to ask ourselves:

  • Do users want a badge for checking into your app every day?
  • Do users want to be publicly ranked at #58,000 on some leaderboard? Will they really be inspired to purchase more to improve their standing?
  • Do users want to deeply engage with your app on a daily basis?

Thinking like a game designer

To take a first-principles approach to gamification is to think like a game designer. Let’s say, for instance, we are a mapping company, and we depend heavily on users to add place listings to our maps. How should we go about encouraging them to do this more often?

The standard gamification approach might say: Let’s add a leaderboard! People like competing and seeing themselves on a leaderboard. The more places they add, the higher up they find themselves on the leaderboard.

But what if we take a game designer’s approach? In this case, we care less about nominally borrowing elements from games and more about whether or not our users will actually respond well to them. We care about whether a feature will realistically make an experience more fun and engaging for the user.

When we describe gamification as “adding game-like elements,” we are automatically boxing ourselves into a process where we identify what these game-like elements are and then force them to fit any non-game context … if we take a [first-principles approach], we care less about nominally borrowing elements from games and more about whether or not our users will actually respond well to them.

Do users care about adding places to a map? Do users want to be better at this action of adding places to a map? Probably not, which means that tools like badges might not be so effective.

Will they be motivated if they are seen on a leaderboard? Maybe, or maybe not.

Do users want to be seen on a leaderboard, where they compete with 100,000 other users and find themselves at the bottom? Would they be inspired to rise up this leaderboard? No.

But what if we put users on a smaller leaderboard, where they’re always relatively close to the top? Would this be more motivating? It’s very possible.

This line of thinking might lead us to conclude that a dynamic leaderboard, where we filter results in such a way that users always find themselves somewhere close to the top, is a good fit for our product. For instance, a user may languish at #58,000 rank on a standard leaderboard. But, competing with users who joined on the same day as him and living in the same residential area, they’re actually now ranked #300. Makes a world of difference, doesn’t it? 

This is exactly what Duolingo does. Every week, you are sorted into a league and ranked on a leaderboard of only 30 people. These are the 30 people who started their Duolingo activity for the week on the same day, same hour, and at approximately at the same level. With this much smaller pool of competitors, there’s much less risk of getting discouraged—and much more incentive to work hard and climb to the top.

By actually considering the user experience and what users are likely to find motivating, we’ve bypassed gamified features that aren’t likely to be much help and arrived at one that might actually work in context.

Too much of a good thing? Understanding game refinement

Here’s another example of a first-principles approach to game thinking. Giving badges is a standard game mechanic; in eCommerce apps in Asia, for example, every few actions you take, you get a new badge. You can well imagine a product manager and designer hard at work thinking of ideas to improve engagement in their app and then deciding to give out badges. You log in every day, you get a badge. You buy something, you get a badge. You buy something every week for a month, you get another badge.  

Sounds great, right? At virtually every turn, the user is given extra motivation to keep using the app. Here’s the problem: turns out, these badges lose their significance quite quickly.

An interesting paper by Huynh and Iida (2017) shares a very unusual insight into game mechanics.3 The authors use a metric called “game refinement,” which is based on the idea that every mechanic embedded in a game somehow “refines” the game, either adding to the user’s enjoyment or detracting from it. These game mechanics can be anything, like profile customization, new types of rewards, leaderboards, streaks, badges, etc. 

The game refinement theory gives a quantitative measure of how much an element “refines” the game it’s in. This is generally given by this equation:

Here, R = the average number of achievements or rewards, and T = the average number of efforts or tasks. How R and T are defined is different for every game mechanic. 

Without going into the mathematics of the formula, let’s take a look at 2 of Duolingo’s mechanics: badges and streaks (the continuous number of days you complete a lesson in the app).  

In the case of badges, the numerator (i.e. the reward) is the badges, and the denominator (i.e. the tasks) is the completed lessons that led to earning each badge. So with time, you are getting the same reward (a badge). As a result, your enjoyment of the badge decreases. Obtaining badges is an exciting activity, but doing so frequently will become boring. That explains why the attractiveness of badges decreases after a certain period. 

In the case of streaks, it is slightly more complicated, because inherently, the longer the streak, the more prestige it carries. Someone on a 3-day streak might not care so much about losing it. Someone on a 427-day streak (like me) is desperate to keep it going—I would even pay to maintain my streak. (Duolingo lets you purchase “streak freezes” that protect your record, even if you technically miss a day.)

Hence, the definition of refinement must take into account the value of the streak at that moment in the formula.  In this case, the numerator—the reward—increases with time, because the psychological weight the user places on this reward does too. Graphically, this is what the measured game refinement looks like for streaks and badges on Duolingo:

If you have used Duolingo as diligently as I have, you would notice that in the first month, you receive badges for every little thing you do. However, as you progress in the app, you receive fewer and fewer, until you’re hardly getting any new badges. This is because the Duolingo team is aware of the principles of game refinement (either explicitly or intuitively). As streaks come to matter more to you, it is less and less necessary to provide appeasements like badges to try to pique your interest—and there’s a greater risk that you might just start to find them annoying.

Please note that none of this has anything to do with whether or not Duolingo is an effective language-learning tool. Many have argued that it’s actually not so useful to help people acquire languages.4 But for the purpose of this article, I am focusing more on the app’s ability to engage users, rather than its ability to teach.

So, how do we do gamification right?

I am of the firm belief that gamification, like anything else in behavioral science, requires nuance. Treating it as a toolbox that has to be applied the same way in every context brings about many pitfalls. In the best case, it might give you a few engaged users. In the worst case, it will give you many annoyed users.

So, where should you start?

  1. Clarify what you want to achieve, and what you want the UX to look like.

Should you gamify everything? Should you just use some game mechanics? It’s a spectrum. Decide where you want to lie on this spectrum.

If you are looking at implicit gamification, you are only looking for actions that make your app feel more natural to customers. For instance, take the action of swiping on a profile, like one does when using Tinder.

Contrast this with Duolingo, on the other hand, which uses an integrated mechanism: the entire app is designed around game mechanics. It is not a game per se, but it is structured like one.

Explicit gamification is when there is an actual game being played. For instance, consider the very famous Alipay Ant Forest, a feature that is separate from the rest of the app. Here, the user plays a game by growing a virtual tree, based on their different actions.

  1. Think about what the user wants.

It is natural for us to start with business objectives. For example, how can I get users to engage more? How can I get users to check in every day? How can I get users to transact more? How can I get users to participate on my platform?

When you are asking these questions, also ask them from a user perspective. Does a user care about engaging more with my app? Does my app offer enough reason for someone to log in every day? Is my use case something that requires users to transact frequently? Does the user benefit from participating on my platform? What rewards does the user care about? Can I get away with just giving them a badge? If you’re not aligned on these, the users will be disengaged and lose interest in your game very fast.

  1. Consider the ethics of your approach.

As in all things behavioral science, ethics are important in gamification. If you don’t do gamification well, you get disengaged users. If you do it too well, you get addicted users. Somewhere in between is the proper balance.

Robinhood, the stock investment app, is currently under the scanner for gamifying investments and misleading people into making wrong decisions about money using game mechanics.5 That’s a headline you don’t want to be a part of. This is where the user view comes in handy.

Final words

Gamification is nuanced. A toolbox approach is a great starting point, but may actually be a very boxed view of what games can really bring to the table. It is best not to treat gamification as a quick-fix solution for business problems, but to rather think of it as an opportunity to really understand human behavior, the way game designers do, and then apply that thinking in a way that’s pleasing to the user and also helpful to the business. Exactly how to go about being strategic about gamification deserves a whole different post.

Of course, it’s only thanks to gamification on Duolingo that I can say now with a French flair, “Tout est bien qui finit bien”—and hope to God I got that right.

How Behavioral Science Can Make Airports Less Miserable

As we approach the holidays and (hopefully) the end of the pandemic, we’re all once again experiencing the joy of stuffing ourselves into small spaces with strangers to travel long distances. If you’re like us, 18 months of space has allowed a shift in perspective, and the ability to see things about crowds we might previously have become habituated to. 

Despite the fact that many airports are designed with incredible care,1 being inside one comes with its share of frustrations. Going through security, boarding the plane, stowing your luggage, and even the boarding pass itself always elicit a momentary thought: There must be a better way to do this.

Fig. 1: Everyone’s favorite part of a vacation. Photo by John Oswald on Unsplash

Behavioral science is often used to look for consistent failure points and provide suggestions2 for how to remediate them, but it can offer predictions even more specific. When dealing with many individuals, or crowds, we can use a technique called agent-based modeling to understand how people interact with one another.

What is agent-based modeling?

Agent-based modeling (ABM) is an algorithmic method of creating multiple independent entities (“agents”) that react to stimuli according to predefined conditions. This allows you to imagine humans not as an amorphous mass, but as predictable elements in an equation. 

Let’s look at an example: Considerable attention has recently been given to pedestrian evacuation in case of emergencies, such as fire, earthquake, tsunamis, or a terrorist attack. As people rush to escape situations like these and converge on a limited number of exits, there’s a big risk of injury or even death as a result of being pushed or trampled by the crowd. Reviews of evacuation videos3 and statistical physics models used to reproduce evacuation behavior have confirmed that the greatest bottlenecks are usually near exits. 

Fig.2: Snapshot of an evacuation simulation of a convention or exposition hall. (From Johansson et al., 2008)

Rather than view the pedestrians in this equation as one undifferentiated, panicking group, agent-based modeling approaches them as a collection of individuals who respond in predictable, rational ways to what’s happening around them. Researchers using this approach have been able to identify some surprising solutions to the problem: they tweaked their models by adding a pillar right in front of the exit and found that it reduced bottlenecks, because it would funnel people into two natural lines.4

While the pillar improved the movement out of the exit, it increased the time-to-exit. Another group of researchers decided that this added time was a hazard in itself, and discovered that by changing the obstacle from a pillar to a wide plane, both the bottleneck and the time were reduced.4 

What’s interesting about this isn’t just that it’s a neat, counterintuitive solution, but that the solution could not necessarily have been discovered by looking only at individuals. Placing obstacles in people’s way should not make them move faster—and for a single person, it probably wouldn’t. But by using an agent-based model, behaviors that were otherwise invisible become apparent.5

How this makes airplane boarding less terrible

This brings us back to airline boarding. Not only is the front-to-back boarding method slow, it’s practically the slowest possible method of boarding a plane.6 It has been shown that even having people board at random would be faster.7 

If we reimagine the passengers as agents, we can model their behaviors like putting luggage into the overhead compartment, getting up to let another passenger into their seat (aka seat interference),8 and so on. Researchers tested variations of this ABM experiment and discovered that the best way to board passengers is to have boarding groups alternate based on both row number and which side of the plane they’re on (see below).

Fig.3: The “Modified Steffan” method. Image courtesy of CGP Grey on YouTube.

This method reduced the number of seat interferences from an average of 91 to 9. In terms of time, that’s shaving 11 or more minutes off the boarding process. This could save airlines roughly $300 per flight, and spare the rest of us a collective couple of million hours of waiting around per year.9 

Why should you care?

You might be asking yourself at this point why any of this matters. Aren’t humans just agents, and all science “agent-based modeling”? To which we would say: yes, absolutely! ABM does indeed copy people, and it allows us to answer large, complex, or otherwise prohibitive questions by algorithmically replicating a close approximation of how humans behave. It’s not usually practical to load a few hundred people off and on an airplane over and over again to test which boarding method works best (although somebody has in fact done that).10 

Applying ABM to real problems about the movement of tourists,11 organizational structures,12 and environmental protection12 allows us to achieve a new level of understanding about aggregate human behavior. Hopefully, this article helps reduce animosity toward any one person next time you’re stuck in foot traffic at a festival (take the time to explain ABM to them, I’m sure they’ll appreciate it!).

Note: ABM is more accessible than you think—there are easy-to-use tools in javascript and python that make implementing these techniques a powerful and relatively simple way to deliver business insights.

Millennials, Money, and Chasing the Middle-Class Dream

When it comes to Millennials and their finances, a lot of media coverage would have you believe that frivolous spending is all you need to know about. One less avocado toast or hipster coffee here and there and all that student debt would be gone. Or, for a lucky few, perhaps your inheritance will bail you out down the road (if it isn’t bailing you out already)—because there is going to be a massive wave of inheritance. The biggest wealth transfer in history is already underway, with Canadian Millennials expected to inherit about $1T in wealth from their parents, the Boomers, in the next 5 years. In the US, estimates range from $30T to over $60T over the coming decade. In the UK, it’s about £1T by 2027.

What gets lost when we get caught up in the memes of frivolous spending is that Millennials are living a very different financial reality than the Gen Xers and Boomers that preceded them. Millennials have different financial priorities, different needs, and different expectations. That’s true of Millennials who are struggling to afford the cost of living anywhere but their parents’ basement, and it’s true of the Millennials who are earning seven (or more) figures annually.

Caught between the far-too-reductive view of impoverished Millennials as they are now, and the not-yet-visible contour of rich Millennials as they later will be, the financial sector is struggling to find an effective way to serve this cohort. That’s why TDL undertook a research project to map out the values and priorities of Millennials. You can read the report here, and TDL would like to acknowledge the generous support of the FP Canada Research Foundation. Assembling a more nuanced portrait of this generation, and developing some practical recommendations for financial institutions and fintechs to provide better service to Millennials, contributes to TDL’s mandate of promoting better outcomes for all members of society.

Want to learn more? Hear Dr. Brooke Struck discuss the research live with practicing financial planners at Financial Planning Week 2021.

The “Good Life”

Millennials are those folks born between 1980 and 1995, making them in their late twenties to early forties now. The oldest remember the Wall coming down; the youngest probably remember the Twin Towers coming down. From the research we’ve conducted, the financial goals of Millennials still show a strong influence of middle-class living circa 1950–1970:

  • Get a good education
  • Get a stable job
  • Get married
  • Buy a house
  • Save (for kids’ education and for retirement)
  • Retire

However, the economic reality that they’re living in has really struggled to provide a solid foundation for Millennials to achieve these things.

  • College and university tuition continue to soar, leaving Millennials with enormous debt coming out of school
  • The economy continues to tilt towards more and more precarious labour, with contract work the norm and gig work constituting a major source of income for more and more people
  • Many Millennials are delaying getting married because of their financial situation
  • In Canada, where house prices didn’t crater following the 2008 financial crisis and low interest continues to inflate the overheated housing market, finding a home to live in is becoming more and more challenging—even just to rent
  • Already saddled with so much debt, and having so little predictability in their income, Millennials find it hard to save and cannot take on high levels of risk (and higher long-term yields they command) lest they need to withdraw funds early

In brief, economic conditions are no longer amenable to most people in society achieving what was considered quite normal in the two previous generations. But expectations have been slower to adapt, which is what’s driving a lot of the tension.

What Millennials want

First and foremost, what Millennials are seeking is financial stability and predictability. They want to feel independent. Perhaps no meme depicts this better than the 30-something Millennial (with more degrees than a thermometer) still living in her parents’ basement.

In this situation, a lot of Millennials are seeking new directions. With the material trappings of middle-class life seeming out of reach, many Millennials are prioritizing self-actualization. A Millennial is more likely than members of previous generations to prioritize action on issues like climate change or social inequality, even if that means that they need to sacrifice a few points of annual growth in their portfolio.

In terms of how Millennials want to interact with financial institutions and fintechs, they expect a more personalized approach. They are seeking a more technologically enhanced offering, but they still see a place for human interaction. That puts pressure on financial planners, investment advisors, and other professionals to evolve their service offerings. A professional can’t survive long in the Millennial world claiming that they’re better with Excel than an AI-powered robo–advisor. People just don’t crunch numbers as well as machines do, so professionals need to pivot their offering into the space where the human touch actually adds more value than an app possibly could.

A couple of proposed solutions

Here are a couple of ways that financial planners in particular can offer higher-value service to Millennial clients. The key here is to recognize that the highest priorities for Millennials are often financial stability and financial confidence. For a more in-depth discussion of these, and for additional strategies, you can check out the full report here.

Stability

For young adults who have grown accustomed to unpredictable wages and slim margins of error, the prospect of financial stability can feel like a distant pipe dream. One solution planners can offer to help them get there is to set them up with a buffer fund. 

The concept of a rainy day fund is well known. But we usually conceive of a rainy day fund as something we hope to never have to draw on: it’s for emergencies, the unexpected. A buffer fund is a slightly different concept. In this case, it’s about evening out much smaller (but more frequent) peaks and valleys in cash flow. For example, you could set up a dedicated bank account to receive various payments from gigs, contracts, gifts, etc., which then pays out a stable amount each month to a more standard checking account, which the client interacts with in ways they’re probably already used to.

In brief, a buffer fund is something that we use to stabilize cash flow on a constant basis because we know that it’s too erratic otherwise (whereas a rainy day fund is used to cover major unexpected gaps). Designing a buffer fund with a client, helping them to set it up and fund it, and coaching them to adapt their behaviors so that the system keeps working in the long term: these are all high-value services that planners can offer, focused on the human and behavioral elements that algorithms, at this stage, are not as well set up to offer.

Confidence

Turning to financial mindsets, even as the dollars-and-cents reality of the client begins to improve, there’s valuable work to be done in helping the client to feel better about their money. Our research revealed that Millennials tend to have a low sense of control over their finances—or, to put it another way (as it is formulated in a handful of studies), Millennials often feel that their finances control them rather than the other way around.

Much of the way that financial planning is incentivized in the current ecosystem is around selling investment or insurance products. This may be a contributor to the lower level of focus that’s placed on the implementation of financial plans. Good practice around implementation—check-ins, structuring complex tasks into bite-sized chunks, providing clarity around next steps, and (especially) showing progress back to the client—can all be very powerful tools for also helping the client to improve their money mindsets, providing some easy wins to demonstrate to clients that they do have control over their money. Once again, we see that a principal avenue for delivering value to clients is to focus on the human and behavioral aspects of financial engagement.

Where to from here: a cheat sheet

If you skipped to the end and read nothing else of this piece, here are a few takeaways that you can start applying starting tomorrow morning in working with Millennials.

  • Don’t make assumptions. Just because a given Millennial is carrying a lot of debt and/or has little savings does not mean that they lack financial education or self-control. Many just ended up in an economy that wasn’t ready to support their success.
  • Don’t be judgy. Millennials tend to care quite deeply about the causes that speak to them. Accepting lower returns or prioritizing career impact over earnings may seem like “irrational” financial behaviour, but money isn’t everything. After all, we can’t have an economy if we don’t have a livable planet.
  • Don’t wait. Many financial professionals are used to working with clients who tend towards the affluent end of the spectrum. A Millennial who may be struggling financially right now may have a very different situation in a few years, as the great wealth transfer really picks up steam. When that happens, there will be enormous competition for their attention. If you build a strong relationship with clients now, you’ll have the inside track when their financial situation improves. Worried that that future payday might never come? Welcome to Millennial existence.

Questionable Research Practices in Behavioral Science, and How to Fix Them

In 2012, Shu, Mazar, Gino, Ariely, and Bazerman published a three-study paper titled, “Signing at the beginning makes ethics salient and decreases dishonest self-reports in comparison to signing at the end.”1 The paper demonstrated that asking people to sign a statement of honest intent before providing information—for example, when submitting an insurance claim—can significantly decrease dishonesty, compared to when people sign such a statement after providing information. 

Since its publication, the paper has received hundreds of citations, becoming influential in the study of dishonest behavior. However, a failed 2020 replication and a viral article posted on the behavioral science blog Data Colada in August 2021 had people questioning the results of the studies and the integrity of the researchers.2

Unsurprisingly, these revelations sent shockwaves across the entirety of behavioral science. This was because the article accused Dan Ariely, one of the leading academics in the field, of data fabrication (an allegation which he has strongly denied).

Why did the authors of the Data Colada post accuse only Ariely of data fabrication, even though he had four other co-authors? This was because Ariely was the only author responsible for the study in question (Study 3). They came to this conclusion because of a number of anomalies in Study 3’s original data set that are difficult to explain as results of anything but deliberate manipulation.

One notable discovery was that Study 3’s data, which purported to look at the number of miles driven by auto insurance customers, had a distribution so uniform as to be statistically impossible. The researchers behind the blog post also examined the spreadsheet containing Study 3’s data and concluded that many data points had been duplicated and lightly edited. 

We still don’t know for sure what happened in this case, and all of the paper’s authors have stated that they were unaware of these anomalies and do not know how they came to be. The purpose of this article is not to summarize everything that has come to light through the Data Colada post, but rather to examine why this type of fraud (if indeed it is fraud) happens in academia, and to explore some structural changes that could improve research practices everywhere.

Why does fraud happen in academia? 

Academia is tough. As a graduate student in the behavioral sciences, I’ve witnessed firsthand the pressure people feel to succeed in this field. One of my professors has told me that he only sleeps four hours each night—and his situation isn’t unique. In an interview with Slate, an anonymous female business professor said she “killed [herself]” when she was working her way up from a mid-level undergraduate university to a top-level faculty job.3 In the process, she alienated her students, annoyed her academic advisor, and sacrificed her health. 

These experiences are not surprising considering the importance of publishing papers in academic journals. Universities frequently use an individual’s number of publications and citations as a measure of their success and therefore, it has become one of the most important criteria for recruitment and advancement.4 This “publish or perish” culture has led researchers to scramble to publish whatever they can manage, instead of spending time to develop research that pushes the frontiers of science forward.4

The increasing competition has even driven some researchers to resort to unethical practices such as salami slicing (splitting the same research into many publishable “slices”), p-hacking (misusing statistical analysis techniques to try to find a publishable result in a data set), and even outright fraud. (It should be noted that not all questionable research practices are deliberate; see the look-elsewhere effect.)

The move toward Open Science

As awareness of these issues in behavioral science grows, the Open Science movement is emerging as a partial solution. Although “there is no single doctrine or paper that definitively captures Open Science,”5 it can broadly be defined as “a set of practices that increase the transparency and accessibility of scientific research.”6

One of the key components of Open Science is pre-registration, which usually occurs before the researcher(s) conduct their study. A research plan, which may include the research question(s), hypotheses, experimental design(s), data collection and analysis plans, etc. are all publicly registered on websites like AsPredicted. 

The goal of pre-registration is for the researcher(s) to be transparent about the purposes and methods of the study. After they pre-register their study, they are not banned from changing their research plan, but rather they should justify and document why they made those changes. This is to prevent researchers from modifying their hypotheses post hoc, to align with any statistically significant results they may have happened to find in their data. 

Many researchers also post their data and code on websites like Open Science Framework and Nature Scientific Data so that others can reproduce it, catch any errors, and conduct alternative/additional analyses. Publicly sharing code and data is also what makes replication studies possible. The team at Data Colada and a group of anonymous researchers were able to discover the fraudulent nature of Study 3’s data because the authors of the 2012 paper publicly posted the data. (With that being said, I personally believe that all of the authors of the 2012 paper subscribe to good research practices. It’s just that something went horribly wrong with Study 3’s data.)  

As a behavioral science student, I am very pleased to know that many respected researchers are supporting the Open Science movement. Francesca Gino, one of the authors of the 2012 paper, said in response to the Data Colada article, “Though very painful, this experience has reinforced my strong commitment to the Open Science movement. As it clearly shows, posting data publicly, pre-registering studies, and conducting replications of prior research is key to scientific progress.”2a

Good things take time

Dr. Peter Higgs won the Nobel Prize in Physics in 2013 for his work on the mass of subatomic particles. On his way to Stockholm to receive the Nobel Prize in 2013, he spoke to The Guardian and said that “Today I wouldn’t get an academic job. It’s as simple as that. I don’t think I would be regarded as productive enough.”7 Even though he was talking about the field of physics, the same goes for the field of behavioral science and the social sciences at large.

Academia seems to have forgotten about the phrase “quality over quantity.” Publishing three high-quality research papers should be seen as just as productive, if not even more productive than publishing ten lower-quality papers. The hypercompetitive culture of the field has caused many researchers to resort to unethical practices like fabricating data in order to preserve their careers. 

As we’ve seen from the testimony of people like the anonymous professor quoted above, these high demands can be destructive to the health and teaching ability of academics. But the negative consequences don’t just stop there. In the last decade, uptake of behavior science by organizations in the public and private sectors has rapidly increased; published insights have become the basis for policy changes, business initiatives, self-help programs, and more. If these results turn out to be fraudulent, it could have huge negative consequences for the decision-making of governments, organizations, and private individuals.

The Open Science movement is a step in the right direction, but it doesn’t get to the root of the problem. There will always be unethical research practices if the “publish or perish” culture persists. In my opinion, hiring and promotion decisions shouldn’t be heavily based on the number of papers published and citations garnered. Universities should consider a different approach, perhaps giving more weight to teaching abilities and student feedback. What kind of changes do you think should be made?