Release 5.0 T6 Metrics Analysis - Preliminary

Michael Simpson

Process Assurance

At the end of release 5.0 week T6 (week six of the RUP Transition Phase), we see an upswing in defect backlog above release 4.2 & 4.1 T6 levels.

We hypothesis this is because End-to-End (E2E) testing started this week. This start is relatively later, in T6, as compared to 4.2 where it started in T2. The result is that many of the defects are revealed later in Transition.

We also notice that our percentage of test cases passing in release 5.0 is lower than it was at this time in 4.2 and 4.1. Moreover the percentage of test case passing in 5.0 has only increased 1% in the past two weeks. The progression of test case passing has stalled.

(Note: Process Assurance previously warned of the risk of a stall due to risking defect-fix turnaround times seen in the previous two weeks.)

And the gap between test cases executed and test cases passing is larger than it was in 4.1 & 4.2 at the same point.

If we weight the defect backlog by the percentage test execution (a normalization so we can compare releases) and further if we re-align the phases based on their implementation date (W1) rather than the start of the Transition Phase (release 5.0 has two more additional weeks of Transition than 4.1 & 4.2 did), we see that the normalized backlog is roughly the same as it was in the previous two releases. This suggests that 5.0’s two extra weeks of Transition have been used and is now roughly in parity with releases 4.2 and 4.1.