Product Backlog and the Cone of Uncertainty

The Scrum burn-down chart is done on an iteration-by-iteration basis. The same concept can be applied to a release and is not shown here. The scope of a release or product is known in Scrum as the “Product Backlog”. 

Uncertainty in Product Backlog is a big risk for the project schedule. The problem is that the full scope of the release can be quite hard to estimate because the requirements are not well-know early on. Confounding this problem is that frequently the release date is a hard deadline. This means that a team must perform an unknown about of work in a fixed amount of time. 

The section on Mitigating Risk from Scope Uncertainty describes the process for estimating a release in a requirements phase. But such a phase may not be undertaken. Development may need to begin as soon as any requirements are available. So how can the scope (in terms of LOE) be managed?

This situation is characterized as the “Cone of Uncertainty.” [Boehms]. See the diagram below. At the beginning of the project (iteration-I1) there is an estimate for the scope but there is a large degree of uncertainty around it, generally skewing more upwards than downwards. This is reflected by the error bar about the I1 point. 

As time goes on, marked by successive iterations, scope is better fleshed out. The estimate for the total scope is refined and the error decreases. By the end of the project, the uncertainty is eliminated and the exact amount of scope (LOE) is known (because it is finished). Draw a line containing the error bars on the graph and you form the triangle known as the “Cone of Uncertainty”.

Confounding the project more is the existence of “unknown unknowns.” This is the scope that is not known to exist at the start of the iteration. Because the requirements have not been thoroughly thought-out, there are things that we simply do not yet realize that we have to do. Since the existence of this scope is not identified, it is not estimated. That is why the error bar around early scope estimates is skewed more up than down, to account for "unknown unknowns."

This is distinguished from “known unknowns” which is scope that is known to exist, but for which requirements are not yet fleshed out. We can estimate them with some uncertainty.

Although uncertainty around the “known unknowns” is often taken into account, hedging for the “unknown unknowns" typically is not. But at some point in the project the "unknown unknowns" are revealed (the requirements breadth is determined) and the known scope of the project increases sharply. This is shown on the graph as the increase between I2 and I3.

At this point, the project is in crisis, although this fact is often not realized until it is too late to mitigate it acceptably.

Typically, project scope and staffing needs are defined at the beginning of the project and never updated. A few iterations later, the known scope has increased but neither resources nor time are increased to match. The result is a late project. This danger often exists in PMP-run projects because the LOE is estimated based on a Preliminary Scope Statement. The mitigation for this is to iteratively re-estimate scope, and feed this back into the PMP processes for Scope/Time/Cost management.

The Mitigating Risk from Scope Uncertainty section describes the estimation process for a requirements phase. During Scrum projects, there is a requirements-effort being undertaken during most of the project. Thus the estimation process defined should be undertaken as part of each iteration planning/preparation effort to continually refine the estimate of the scope of the release. 

Identifying the Need for Scope or Staffing Adjustments

In the example being discussed, the four development iterations of the project were planned to achieve the level of scope defined at I1 and the project was staffed accordingly. This is shown below by the rising black line that turns into a red line. Assume we are currently at the beginning of I3, where the “unknown unknowns” have been uncovered and a new level of release scope has been identified on the Y-axis at the I3 mark.

According to the trend, the team will complete the I1-estimated level of scope by the end of I4 (start of I5), but we now (I3) see that it is not enough. Now, at the beginning of I3, the team realizes that it needs to complete the I3-estimated level of scope. To finish the job with the same resources (red line) it looks like it will take an extra iteration. The change in known scope results in a schedule variance showing that the project is projected to be late. 

To rectify the situation, the team has three main choices:

1. Extend the length of the project schedule
2. Remove scope from the product to bring the total scope back to the original level
3. Add staff to increase its LOE capacity for its subsequent iterations to reach the new level of scope

Schedule Adjustment

The simplest solution is to simply extend the project schedule by the schedule variance. How easy this is depends on the restrictions set in the project charter. If there is no strict release deadline then the team is within its rights to unilaterally decide on a schedule change (although it should be negotiated with stakeholders). However, if the project charter constrains the schedule, then the team must go to the customer with a proposal to amend the charter’s schedule constraints.

Scope Adjustment

Removal of scope typically comes in the form of:

1. Removal of some low priority feature sets (take something off the plate)
2. Removal of some low priority features from one or more feature sets (trim the fat)

Feature sets or “themes” should be prioritized by the customer. Within those feature sets, each feature should also be prioritized. A high priority feature set may contain some low-priority nice-to-have bells and whistles that are good candidates for elimination. The Requirements Lead should be able to propose feature cuts to the customer that will not jeopardize satisfying the Key Success Criteria defined in the charter. If this is not possible, the team will need to go to the customer to request an amendment to the charter to formally remove some scope.

An alternative to cutting features is deferring them to after the official release date. If the customer is committed to a release date and to the entire scope, he may be satisfied with receiving maintenance releases after the official deployment date that deliver the deferred low priority / non-critical features. The team may need to go to the customer to officially extent the schedule of the project because resources now need to be committed pasts the release date.

Staffing Adjustment

As a general assumption, an increase in staffing increases the rate at which the team implements product scope and so accelerates the schedule. The result of a staffing adjustment is shown in the previous diagram as the green line with a steeper slope that reaches the I3-projected level of scope by the end of I4 (start of I5). 

There are exceptions to this rule. First, there may not be enough parallelism in the development tasks to enable more people to work on the product. Too many people working in a kitchen results in people getting in each others' way and actually slows things down. Second, anytime a new team member joins a team, other team members have to help him ramp up to being productive. Adding a team member is initially a distraction for other team members and it initially slows down progress prior to increasing it. This is known as the Mythical Man Month effect [Brooks].

Before adding staff, the team should carefully consider the ramifications. If staffing adjustments are to be made, generally sooner is better. The earlier more staff is added, the more iterations are available to implement scope at an accelerated rate. Regarding the Mythical Man Month effect, a new team member may take an iteration to ramp up, so the more iterations that follow the initial one, the more of a return on investment can be achieved. Never add development staff in the last couple iterations because progress can only slow down as new ramping-up team member tax the existing staff. Conversely, the very first iteration of a project is best run with fewer “cooks in the kitchen” who can lay down framework and pave the way for more people to be able to work in parallel.

Priority should always be placed on getting a handle on the breadth of the requirements (eliminating unknown unknowns) so that everything that is on the plate is known. With that, everything can be estimated and correct staffing decisions can be made early. Include a calculated buffer time to account for known-unknown uncertainty as described in the estimation section.

Recommendations

In anticipation of scope problems, the development schedule should be aligned to address the highest priority feature sets and features in early iterations, and defer lower priority feature sets and features to later iterations. To reduce risk around missing a release deadline further, the Requirements Lead should activity defer low priority features from all feature sets to later iterations. This avoids the case of being unable to release a minimally acceptable product because time was spent on low priority features from high priority feature sets when it could have been spent on all individually high priority features across feature sets.