Why Software Projects Fail: Causes and Early Warning Signs

Bent Flyvbjerg and Alexander Budzier studied 1,471 IT projects for Harvard Business Review in 2011. The headline number looked manageable: an average cost overrun of 27 percent. The finding that mattered was hidden in the tail. One project in six was what the authors called a black swan, overrunning its budget by an average of 200 percent and its schedule by almost 70 percent.

That’s the uncomfortable truth about software failure. Most projects don’t collapse. They finish late, over budget and with less value than promised. The ones that do collapse can take a company with them.

What the research shows, and what it doesn’t

The most quoted source is the Standish Group’s CHAOS Report. The figures usually quoted for 2020 are roughly 31 percent successful, 50 percent challenged and 19 percent failed. The report itself is paywalled, so they are hard to check in public. They still appear in nearly every slide deck on the subject.

Handle them carefully. Robert Glass asked in Communications of the ACM back in 2006 whether the report really proved a software crisis, since other studies found very different results. That same year Magne Jørgensen and Kjetil Moløkken-Østvold showed that the 1994 report’s famous 189 percent average cost overrun was probably far too high, possibly because respondents were asked for stories of failure. J. Laurenz Eveleens and Chris Verhoef criticised the success definitions in IEEE Software in 2010. The raw data has never been published, and the definition of success changed in 2015, so year-on-year comparisons are close to meaningless.

Studies with published methods are more useful. Here is one of the largest:

The value shortfall deserves more attention than the cost overrun. Delivering 56 percent less value than planned means that even projects landing on budget often failed to deliver what they were started for. That argues for smaller steps with early checks on whether anyone needs what’s being built, the core case for iterative delivery covered in agile vs waterfall.

The UK’s National Programme for IT shows what the tail looks like at national scale. Launched in 2002 to digitise patient records across the NHS in England, it was announced for dismantling in 2011. Two years later Parliament’s Public Accounts Committee reported that the NHS still expected to pay around £9.8 billion for it.

The seven most common causes

Studies, inquiry reports such as the one on the NHS programme, and the project management literature point to recurring patterns:

1. No measurable goal. Without a definition of success, nothing can be prioritised; every requirement looks equally urgent to someone.
2. Scope that grows quietly. Many small additions, none formally assessed. Together, a different project.
3. Estimates become commitments. An early number named under pressure turns into a deadline, although the cone of uncertainty says it could be off by a factor of four.
4. No decision-makers on the business side. The team waits for answers or builds on guesses that get reversed.
5. Sliced too big. Projects that deliver nothing for two years pile up risk without ever testing it.
6. Underestimated integration. Connecting to legacy systems and migrating data get treated as side issues, then dominate the effort.
7. No stop rule. If nobody agreed when to stop, nobody stops. Money already spent distorts every later decision.

Notice what’s missing. “The developers couldn’t code” rarely shows up as a root cause in these analyses. Most causes sit before the first commit or outside the development team.

Warning signs to check every week

Crises announce themselves. None of these signals needs specialist knowledge to spot:

Two or more at once is reason to call the steering group now, not at the next scheduled meeting.

Countermeasures that work

Here’s the good news: almost every cause has a simple, unglamorous countermeasure.

None of this is new. It still rarely gets done consistently, because it costs time at the start, exactly when everyone wants to get going. How to build it into preparation is covered in how to plan a software project, and why ranges beat single numbers is explained in our guide to software estimation techniques.

When the project is already off the rails

Troubled projects need honesty about their real state first, not more people. Fred Brooks noted in 1975 that adding people to a late software project makes it later. Three steps work better: establish how much scope is genuinely accepted, re-estimate the remainder, then choose between cutting scope, extending the envelope or stopping.

Stopping isn’t defeat when carrying on costs more than it returns. That’s hard to accept once real money has gone in, which is exactly why the stop rule belongs at the beginning, before it hurt anyone.

An outside view can help. An independent technical review, for instance by a studio such as OnlineBuilders, can establish whether the architecture and code can carry the project forward or whether a restart would be cheaper.

Next step: run the warning-sign checklist against your current project. If you hesitate on any line, that hesitation is your answer.