I have a lot of respect for Michael Bolton (the testing consultant, not the singer). And I think he’s mostly right in his questioning of the accepted relationship between stage of the lifecycle and the cost of finding and fixing a defect. Barry Boehm’s findings at TRW in the 1980s shouldn’t be taken as a hard and fast rule. The most common expression of his research findings is that the cost of finding and fixing a defect increases by an order of magnitude at each stage of the development lifecycle.
But…
Michael’s examples are intentionally trivialized. It is certainly possible that a defect found just before product release can be shown to a developer and fixed in a few minutes. But unless the fixed code was completely unrelated to anything else in the code base, there should at least be some regression testing done prior to release. In regulated or safety-critical products, teams will also have to prove that the fix fully addressed the defect.
And most teams focusing on process improvement would want to understand why the defect appeared when it did. Was the process at fault, or were there other changes to the code that uncovered this particular defect? Were there test cases blocked until the very end, or did the team not have good test coverage?
Even in the trivial cases, it is possible to see that regression testing and root cause analysis take more time later in the development lifecycle. That’s especially true if the final product is an integration of different components and libraries, which increases the overall complexity of the product once integration testing is complete.
In the vast majority of cases, a defect found later in the development lifecycle is more subtle and complex than average, and usually isn’t a simple forgotten statement or mistyping. It’s very likely that the testers have found and tracked all of the defects that can be fixed in just a few minutes during the early parts of testing. What is left is more difficult issues that require creating testing and extensive defect analysis before a fix can be implemented. After that, it still requires regression testing and likely even some integration testing to verify the fix.
He’s right that we shouldn’t be slaves to the idea that finding and fixing a defect later is always more costly. But there is no good reason to use that fact to become complacent about finding and fixing defects early. If teams can leverage automation and process tools to find the difficult defects earlier, we are doing ourselves and our project a big favor.
The post What is the Cost of a Defect? appeared first on Blog.
