thank you for your interesting links. I believe that all you say is true.

I was trying to make a different point, though. I tried to extract the effect of one practice with regards to the defect rate and the expected cost of a defect.

I include the worst-case up-front additional cost for introducing a new practice in the coding effort factor. I use a worst-case defect-rate improvement to extract a pi times thumb number for the worst case cost of a defect for which the practice would be cost efficient /just by looking at it’s impact on the defect rate/.

If it is already cost efficient when I look at the defect rate, then I don’t need to look further. The effects I ignore can only /improve/ the practice’s pay-off, this is why I ignore them - because I believe that in many environments a small impact on the defect rate alone improves productivity.

The interesting point you make is the idea of time-to-market. One could argue that this cannot possibly be included in the “worst-case” part of my crude model. On the other hand I don’t yet believe that you really “loose” time to market with a lower defect rate, since most probably the customer will find some of the errors and demands that you fix them before going life.

Cheers,
Manuel

Often, developers understand “productivity” to be a measure of the quantity of source code they can produce in a given time. If we take “productivity” to mean the amount of fully-tested, working code (that meets the acceptance criteria) they can produce in a given time, then the effects of TDD and pairing become more evident.

In a traditional approach to programming, a developer types as much code as possible as fast as possible, and then spends a certain amount of time testing (usually manually and informally) and debugging the code. It’s normal for the testing and debugging time to be much greater than the original coding time.

Using TDD and pairing, the initial testing (as well as the detailed technical design) occurs concurrently with coding. In accordance with the well-known rule of thumb that it is cheaper and easier to fix errors when they are found early in the process than when they are found late, the original code delivered in this way tends to have a lower defect density than code delivered in the traditional way.

There have been a few controlled studies of the effects of TDD and pairing. George Dinwiddie has kindly set up a wiki where people can post links to studies that can help us explain the value of agile methods. The wiki is here: http://biblio.gdinwiddie.com/biblio. You might find information there that will help your team understand the costs and benefits of TDD and pairing in an objective way.

One of the best-known studies of pairing was conducted at the University of Utah in 2000. Researchers found that the initial development time increased by about 15%, and the number of defects was reduced by 85%. A 1975 study by the US Army of so-called “two-person team” programming found a three-fold increase in “productivity” at no cost in development time.

Taking the 15% figure as a worst-case baseline, let’s compare what happens in traditional and agile development, and see which approach takes less time to deliver fully-tested, working code that meets acceptance criteria ( = “is more productive”).

For purposes of illustration, I will use the adjusted defect density rate found in this study http://www.idi.ntnu.no/grupper/su/publ/pdf/ericsson-qa-icse04-final.pdf for non-reused code, 0.631 per KLOC. I will also accept the result from the Utah study of pairing, that this practice adds 15% to the original coding time.

Traditional: Four developers work separately (no pairing) and write code first (no TDD). Each individual produces 10,000 lines of code in the amount of time scheduled for development. After that each, still working separately, performs manual, informal testing of their own code. Next, the four developers integrate the modules they have written. This integration step adds perhaps 25% to the time spent in development.

Agile: Four developers use the agile techniques of promiscuous pairing (they switch with each other during development) and TDD, including disciplined test-first development with incremental refactoring. The four collectively produce 40,000 lines of code in the same fixed amount of time as the traditional team. Thus, the two teams produce the same raw quantity of original source code.

Notice that there is no need for after-the-fact integration of the code developed in the agile way. In effect, then, the traditional approach cost 25% more than the original coding time, and pairing cost only 15% more than solo coding.

Now the two teams are ready for formal QA testing. They submit their software to the company’s QA testing group for formal testing.

The QA group finds about 25 defects in the traditional team’s code. The development team is unaware of these bugs, and must fix each one as the QA group discovers and reports it. This may take 3x the original coding time.

Ignoring TDD for the moment and taking the 85% reduction in defect rate that results from pairing, the QA group finds 4 defects in the agile team’s code. The agile team took 15% longer to produce the original code, and required a small amount of time to fix 4 defects. In total this may have been 2x the original coding time of the traditional team. In a nutshell, counting after-the-fact integration time, the traditional team has taken more than 2x as long as the agile team to deliver the same solution.

Note that a combination of agile techniques used together may have eliminated all defects before the software was submitted to the QA group. A clear definition of “done” and frequent feedback from customers complement practices like TDD and pairing to improve quality still further. It is not unusual for agile teams to deliver code to QA that has zero defects.

All of this ignores the effect of technical debt on the cost of a project and on the cost of supporting the resulting product. Because of incremental refactoring, TDD prevents the accumulation of code debt. That means modifications and support are much easier and cheaper for a product that is developed in this way than for a product developed in the traditional way.

The question becomes, Which is more costly: 15% additional time for the original coding, or 2.25 X the time-to-market, plus increased support costs over the production lifetime of the product?

Which team has really been more “productive?”