A close-up view of a classical government building with large marble columns and an engraved eagle at the top.

Why Government Projects Fail: A Behavioral Analysis

read time - icon

2 min read

Oct 21, 2024

It is a well-known fact that governments often fail to deliver the things they set out to achieve. New schemes, programs, and projects are often rolled out over time or over budget—or just don’t quite live up to their original promise. 

As the directors at a specialist government advisory firm called PUBLIC, we’ve witnessed these shortcomings firsthand. PUBLIC works with the UK and European governments to deliver large, complex digital, data, and technology projects. From our experience, we know that things aren’t always as smooth as originally anticipated. The real question is: why?

In this article, we’ll look at some of the common failures in government projects through a behavioral lens. We believe that cognitive biases, habits, and heuristics are often the root of the problem and that by being more aware of them, governments can get better at fulfilling their promises. We’ll investigate seven examples of project biases, and offer strategies to overcome them—drawing inspiration from Kahneman’s recommended “decision hygiene practices” to keep biases in check.

As digital practitioners, we’ll focus mainly on digital and technology projects, but we believe our principles are true for any kind of project—including infrastructure, housing, healthcare, or anything that requires careful planning and even more careful roll-out. 

1. We tend to think our projects are more unique than they actually are

Even for the most cookie-cutter projects, we usually think our case is somehow different or more special than others. We see this all the time, especially when it comes to software. Most digital systems that governments build are basically the same as the rest, all boiling down to a single database or a simple webpage. But even when something has been done thousands of times before, it can still be tempting to think: we need something a bit different, because of how we’re set up, or, we might need to change the user interface around, or, that doesn’t quite fit how we do things.

There’s strong recent evidence from Danish economic geographer Bent Flyvbjerg and others that the uniqueness bias—the perception that our projects are different when they simply just aren’t—is the single biggest predictor of major projects failing. Perceived uniqueness can be a problem for a few reasons. First, it can cause inefficiency: teams try to reinvent the wheel (or, in this case, established processes) from scratch or try to solve problems that already have reliable solutions at hand. Second, it can mean that teams miss out on learning the lessons from other projects that have failed, which, as Flyvberg’s study puts it, “results in a distorted view of reality that underestimates risk and overestimates opportunity.”

Instead of assuming uniqueness from the get-go, teams should start from the premise that their project has been done before, and therefore, should try to gather whatever insights they can from the previous work that came before them. Getting this all out in the open before a project starts can help the team come to an important realization: their project is probably not one-of-a-kind, and already has a blueprint they can follow and refine to their own needs.

2. We are too optimistic when planning projects

Optimism bias refers to our tendency to be too optimistic when planning or accounting for future outcomes. This effect becomes even more extreme as we look further into the future—which is always needed for complicated government projects. Building a new highway, rolling out a new healthcare plan to millions of people, or developing a brand new piece of software all take time, and all require forecasting years down the line.

In the UK Government, we actually have a standard process for applying an optimism discount to financial forecasts. This means that, when planning projects, teams are required to add additional charges to their future financial projections when estimating how much a project will cost. Usually, this is an extra 10-20% of the total cost, depending on the project. 

But the optimism bias is not something that we just need to be wary of at the beginning when making grand financial projections. It’s something that can impact decision-making at any given moment. When teams get together to plan their tasks for the day, or the week, or the month, they regularly underestimate how complicated these tasks are, or how long they will take. 

A good hygiene process when planning or dividing up tasks is to check yourself for potential optimism bias. It’s safe to assume right off the bat that things might take more time to get done than you initially planned. A simple rule of thumb might be increasing the estimated time by 20-25% when projecting how long something might take you in the future. 

Another helpful strategy to check your optimism bias is teaming up with your colleagues to conduct a project pre-mortem which involves considering all of the things that could go wrong before you get started, rather than after. This can help to dial down your optimism, as it forces you to face up to the things that might not unfold as expected.

3. We have poorly evidenced anchors for how long things should take and cost

Projects usually start with a bit of loose scoping. In the technology sector, sometimes we call this “t-shirt sizing”—or, as a translation, we broadly work out the size and shape of the project, before sweating over the details. But unfortunately, this process is highly susceptible to risks of the anchoring bias.

The anchoring bias is a cognitive effect that causes us to prioritize recent or salient information (“the anchor”) when making a judgment or decision. Anchors can influence our perception of both the cost and time associated with a project. When drawing up a budget, building a financial model, or estimating project timelines, teams could be anchored by a range of different pieces of information. Perhaps this is a quoted price someone said in a meeting a few weeks ago. Or how much it cost last time to complete the project. Or how many weeks the director expects it to take. Sometimes, the anchor can even come from external sources, like how much a vendor or consultant told you it would cost. 

All of these anchors can get in the way of objective project planning from the ground up. Teams often don’t realize that when they’re planning or costing out a new project, they’re often using the original anchor as the “jumping off” point. This means that they aren’t building their plans or budgets according to the specific details of a project, but according to numbers they might have been told, instructed, or plucked out of the sky.

The best way to mitigate this risk is to lay out all of your team’s anchors at the beginning of the project and assess how objective they are, along with how much weight you should actually give to them.

behavior change 101

Start your behavior change journey at the right place

4. We prioritize looking good rather than doing good

We have an innate tendency to prioritize achievements that are more visible to others, rather than the most important thing to get done. It’s hardly surprising that when we’re working in a team, with senior leaders checking our work, we want to look good in front of our peers.

In a software context, a boring but important process of automated testing is a great example of this. Automated testing is when we use tools to test for bugs, errors, or defects in software. This is considered very much behind-the-scenes work: outside of the technical team doing these tests, nobody even knows it’s happening. It is certainly not visible to management or to customers. Often, when time is limited, teams will deprioritize these quality parameters in favor of doing the things that people can see: building new features, making the user experience more aesthetic, or addressing senior leadership requests. 

This is a case of prioritizing looking good over doing the most important (but often invisible) thing. A well-informed project team would likely sacrifice new features and upgrades or even extend project budgets to ensure this hidden but essential work gets done. But in most circumstances, teams might have these priorities swapped.

To tackle this risk, it’s important that project teams actively celebrate and incentivize the less visible project achievements, as much as the visible ones. This means getting senior leadership excited about the more “boring” but critical parts of a project, or even showcasing this behind-the-scenes work to potential clientele.

5. We don’t understand our hopes, fears, and drivers in projects—or those of our teammates

Project planning often lays out what we want to achieve, and, if done well, how we want to achieve it. Plans consist of numbers, costs, resources, dates, and milestones—but usually fail to properly account for the people involved themselves. 

Specifically, we don’t consider things like: why is this project important to me? Why am I spending my time on this? What fears or anxieties might be driving me? How do I feel when I think about the project? And if we are often unable to answer these questions about ourselves, it’s no surprise that we usually don’t understand what’s driving our team members, either.

Unfortunately, it’s understandable why these questions are not treated as a priority alongside the cold hard facts of a project. When there is work to be done and deadlines to be met, it’s natural that we jump straight to the numbers and spreadsheets. But the human drivers of projects are really important, too. 

If we take the time to understand our team’s motivations at the beginning of the project, we will be able to better understand the role that our emotions are playing in shaping our project decisions—as well as those of our colleagues. For example, we might more easily identify when team members are making decisions driven by fear, guilt, or pride. There is an established field of literature focused on how emotions can impact our risk tolerance in different situations (behavioral economist George Loewenstein described this as risk as feelings theory). 

This can all be particularly useful when we experience bumps in the road in our projects (which is pretty much unavoidable). These are the kinds of high-stress moments when emotional decision-making can take over. A team that understands each other’s hopes, fears, and drivers will be better placed to navigate these difficult situations and improve their collective decision-making—and hence, the project outcomes that follow.

6. We fail to align incentives, especially with partners and contractors

Many projects do not have the right overall incentive structures in place to facilitate success. In other words, the overall success of the project does not always align perfectly with the success of all of the project team members involved. This is particularly true of projects involving external partners, contractors, vendors, or consultants, who may be rewarded even if outcomes are not positive.

A common example of this phenomenon lies in software development. When writing code, most developers are not typically incentivized to ensure that their code is easy to use, navigate, or update in the future. Instead, they’re usually incentivized to produce the quickest output possible, even if that means shortcuts or cutting corners on quality. This is because, usually, the person writing the code is not the person who’ll be maintaining or updating it in a few years’ time.

This is especially the case if there’s an external vendor or contractor writing the code, which is very common for government projects. In most cases, external partners are not only incentivized to make things easier for the future—they may actually be incentivized to make things harder for the future so that their clients keep them around. 

Luckily, this doesn’t have to be the case if incentives are aligned from the start. The important thing here is to ensure that partners are motivated by the success of the project itself. As much as possible, project members—whether they be internal or external—should be incentivized by outcomes, which especially includes making it simple for someone to pick things up where they left off.

7. We sink more resources into the wrong projects

Finally, the classic project management problem. Governments continue to sink time and resources into the wrong things, even when it would be a better decision to stop while they’re behind. This is a phenomenon known as the sunk cost fallacy: our tendency to follow through with something that we’ve already invested heavily in (be it time, money, effort, or emotional energy), despite obvious signs that giving up is a better idea. 

In software development, we see this all the time. There are lots of examples of public bodies in the UK that have spent—and continue to spend—millions on technology projects that they know are not going to work. They do this primarily because they have sunk so much time into them, or wasted so much money, that it feels impossible for the organization to admit to itself that it hasn’t worked. Even worse, it’s pretty common that a government department will be spending money on two identical systems: one that works, and one that doesn’t, because they can’t bring themselves to close one of them down. 

We think the answer here comes down to more regular, and more honest, appraisals of our projects. If we let things run unchecked for years, we might find that we have been sinking money into things that we all know just aren’t worth it. Another important step to mitigate sunk costs is to reframe decisions to cancel current projects positively as ‘future savings.’ For example, if we cancel a live technology service, it may have cost $2 million to develop, but we have saved over $10 million in future lifetime costs.

Checking your behavioral blindspots

These are just some of the hidden behavioral factors that can get in the way of government projects being done right. Most of them are blindspots we might already be aware of but fail to account for in our day-to-day working practices. The decision hygiene steps we have outlined are quick checks that can be done before a project, as well as throughout its delivery. Some of these small actions could make a major difference in the long run.

Remember that the stakes of getting this right are really high: these kinds of mistakes can cost literally hundreds of millions of dollars, and stop important things from getting done. But by starting to recognize these biases, we can transform project failures into lasting success.

About the Authors

A person with short red hair and a beard is smiling in front of a dark blue wall. There is a large green plant and a decorative architectural element in the background.

Johnny Hugill

Director at PUBLIC

Johnny Hugill is a Director at PUBLIC, where he supports public authorities to deliver and evaluate major projects. He has been a member of a number of UK Government advisory boards, including for public procurement. He has previously written for The Decision Lab and other behavioral science outlets about the intersection of behavioral science and government.

A person with short light brown hair and a beard is standing in front of a dark blue wall, wearing a plaid shirt. There is a green plant and a decorative ceramic vase on a mantelpiece in the background, creating a warm and cozy atmosphere.

Richard Llewellyn

Director at PUBLIC

Richard Llewellyn is a Director at PUBLIC, leading its Digital, Data, and Technology projects with public authorities. He joined PUBLIC after leading product development at a HealthTech startup called Eva Health, having previously led product development at a range of tech scale-ups.

About us

We are the leading applied research & innovation consultancy

Our insights are leveraged by the most ambitious organizations

Image

I was blown away with their application and translation of behavioral science into practice. They took a very complex ecosystem and created a series of interventions using an innovative mix of the latest research and creative client co-creation. I was so impressed at the final product they created, which was hugely comprehensive despite the large scope of the client being of the world's most far-reaching and best known consumer brands. I'm excited to see what we can create together in the future.

Heather McKee

BEHAVIORAL SCIENTIST

GLOBAL COFFEEHOUSE CHAIN PROJECT

OUR CLIENT SUCCESS

$0M

Annual Revenue Increase

By launching a behavioral science practice at the core of the organization, we helped one of the largest insurers in North America realize $30M increase in annual revenue.

0%

Increase in Monthly Users

By redesigning North America's first national digital platform for mental health, we achieved a 52% lift in monthly users and an 83% improvement on clinical assessment.

0%

Reduction In Design Time

By designing a new process and getting buy-in from the C-Suite team, we helped one of the largest smartphone manufacturers in the world reduce software design time by 75%.

0%

Reduction in Client Drop-Off

By implementing targeted nudges based on proactive interventions, we reduced drop-off rates for 450,000 clients belonging to USA's oldest debt consolidation organizations by 46%

Read Next

Insight

Supporting Mental Health on College Campuses

College students are struggling with rising mental health challenges, from overwhelming academic pressure to long wait times for counseling services. While many universities are scaling up support, traditional approaches often fail to reach students who don’t seek formal therapy. By leveraging behavioral science, universities can implement scalable solutions—like stepped care models, resilience programs, and peer-led initiatives—to provide more accessible and effective mental health support.

Insight

How to Preserve Agency in an AI-Driven Future

Learn how to take back control in an increasingly automated world. This article explores the challenges automation poses to autonomy, the importance of meaningful decision-making, and actionable strategies like human-centric AI design, agency-preserving workflows, and AI literacy programs.

Insight

Why Are We Polite to ChatGPT?

Explore why we instinctively say "please" and "thank you" to ChatGPT, even though it's just a chatbot. Learn how politeness affects AI responses and how our interactions with chatbots can shape workplace culture and human etiquette.

Notes illustration

Eager to learn about how behavioral science can help your organization?