Innovation Trends

Why we Should Rethink the Agile Manifesto: Projects Still Fail

This is the fifth post in a series examining the changes that have occurred since the Agile Manifesto was published and the implications they have on how we might consider the Manifesto today. Find the first post here

In the previous posts in this series I discussed that the world has changedsoftware is now everywhere, and complexity has increased. Today we talk about the fact that in spite of the promise of Agile, projects still fail. Why is that? Why can’t we get to a point of 100% success?

One study, published in 2012 by Dr.Dobbs reported that Agile had a 72% success rate, compared to a 64% success using traditional methodologies. While better, an 8% improvement is hardly a revolution. In today’s competitive business environment, we need to do better in terms of success rate. A further study, conducted by McKinsey, reported that half of IT projects with budgets of over $15 million dollars run 45% over budget and deliver 56% less functionality than predicted.

Put simply, Agile is not a silver bullet. Projects still fail at roughly the same rate today as in 2001. It seems little has changed or evolved in this respect. Looking beyond the numbers, there are several reasons why more than one in four projects fail:

  • Teams are out of sync. This is not just about development teams. It’s about cross-departmental teams, from business to engineering, executive to employee. It’s about the company’s entire ecosystem. An engineering team being agile only makes them agile but does not make the company agile. The process or methodology selected should take into consideration the entire organization.
  • Secondly, as we’ve alluded to – there really has not been any silver bullets. This is an historic reason, but nonetheless still relevant. The problem is that we continue to seek out the silver bullets – the ‘killer app’ – and fail to recognize the nuances of human behavior and complexity.
  • Decisions, decisions, decisions – Decisions have an impact, and there are hundreds, if not thousands, of decisions that take place during a product lifecycle. Robert Goatham, in his paper The Story Behind The High Failure Rates In The IT Industry outlines the number of individuals involved compounded with the complexity of the project as primary reasons IT projects fail more frequently than others.
  • Lastly, people matter. This is a cliché, but it is 100% true. You could have the best process in the world and if the people are not invested, or don’t care or don’t get along, you are at risk. It’s not to say that people are the only thing that matter but it’s a very important factor in a project’s creativity, innovation and success.

In addition to the reasons mentioned above, the whole idea of failure has shifted. The Agile Manifesto promoted the idea of working software over big bang releases, but working software can have different meanings in organizations. Defining product success is vital. It’s important to establish what it means in yours. Evaluate the correct amount of context you need to provide your stakeholders to ensure you can maintain a working software process. Working software is no longer about just getting half-baked increments to the customer with a means for review. Moreover, it’s about providing something viable; something tangible and which adds true value.

Even failure can provide useful data. If analyzed correctly, the data collected from failure can be valuable and actionable feedback. Product management is not only about defining what needs to be built but also about designing customer and stakeholder expectations. By defining outcomes and managing expectations with the rest of the organization, you’ll be better able to utilize the data obtained from product failures.

Success is theoretically about getting it right. Yes, agile and the idea of working software is about putting it out there for feedback, which means you are not always going to be right. What’s key is to be prepared to react to the feedback so you can act quickly and continually. This builds loyalty and admiration in a world where customers are highly engaged and demand interaction.

Check back next week for the next part in this blog series. In the meantime, find me on Twitter or share your thoughts on the Manifesto in the comments below and download my eBook, “A Modern Take on the Agile Manifesto.”