Why Software Project Still Founder and Fail in an Agile World

We all know good project management relies on the managing the triple constraint: Scope, Schedule and Cost. Most software projects that fail or underperform do not do so because of poor management of the triple constraint. Instead, they founder because productivity diminishes over time, often slowly and imperceptively.

By productivity, we mean the rate at which a team can deliver business value to applications. This would seem like a simple thing to measure but in fact it is quite elusive. First of all, what "value" means can be subjective. Add to that the cost of two things of the same business value can be quite different. Based on the state of existing software system architecture, design and tools one thing may be easy to deliver while another is harder.

Agile software methodologies have taken steps forward to address this. Notably, in XP defines a point system and Scrum as "backlog." But this system is aimed at capturing the relative costs of the business value "stories." It does not directly quantify the business value of the stories (only relative priorities). To start with, managers/customers should quantify the value in some terms. This gives them the opportunity to see the value-per-cost ratio.

Most managers do not do such a formal calculation, but do have a sense of it in their head. Typically, they complain that over time, they don't feel like they are getting the same amount of value out of a development iteration that they used to. They look at the "points" being delivered and it is the same, but they don't feel like they are getting as much value. 

Frequently, the problem is a design deficit or quality deficit. This means that the existing code infrastructure is inadequate to support the productive development of new functionality. But how do we know when and how much to invest in refactoring for increased productivity, especially when we don't have a quantitative measure of the reduction of productivity?

Its the Economy, Stupid (but not the one you are thinking of)

In part, management does need to quantify business value, but that is only part of the story. In one project, a set of teams practicing XP delivered 100 points of functionality in an iteration. But the pressure from top management was on. So the project management called a meeting and asked the teams to "dig deep" in the next iteration and deliver 10% more points. Did they do it? They sure did. Was it 10% more value? Absolutely not.

Points are like a currency. In they case demand for them rises, but the supply is limited. Classic economics says the price should go up. But management is not supplying more resources to pay for them (time, staff) while the staff (contractors in this case) are under great pressure to satisfy the customer. So what gives? Substance. The point estimates on the "stories" were inflated. Or in other words, the value of the point currency deflated.

When you are valuing something in a currency, the true value of which is a moving target, then you can't make any legitimate comparisons. If points this iteration are not the same as points the next iteration, and points on one team is different than those off another, then 1-to-1 comparisons are meaningless.

To deal with this, we do these things:

1. Normalize business value and LOE cost estimation across teams
2. Protect estimators from economic pressures
3. Track inflation/deflation over time
4. Track changes to productivity based on the factors of production
   

Image you are not a project manager but the Central Banker of your own little economy. You will need to exercise good monetary policy that encourages price stability. What's more, you will also need to track inflation/deflation. The Federal Reserve does this by sampling the prices of commonly purchased items over time. Similarly, you will need to sample the point estimates of fairly well-understood development tasks over time. If you have kept your currency stable, and see a rise in point estimates, you can then relate that to how well the infrastructure supports these activities.

If your currency is stable and overall productivity is declining (Gross Domestic Product), then you may be in a recession that require a stimulus in the form of an investment in refactoring to sure up design and quality and set the economy back on track.

The Simpson Software Studio approach is aimed at making you that good Central Banker.