EWD249, Damn-Near-Zero, Why Unit Tests Are Necessary, and Why Testing Is Not Enough

Recalling that our true concern is with really large programs, we observe, as an aside that the size itself requires a high confidence level for the individual program components. If the chance of correctness of an individual component equals p, the chance of correctness of a whole program, composed of N such components, is something like

P = p^N

As N will be very large, p should be very, very close to 1 if we desire P to differ significantly from zero! --Dijkstra / EWD249 (pp. 9 - 10)

 In other words, if you are writing a very large program composed of many components, each component needs to be near perfect if the whole program is to have any semblance of correctness. The reason is because of the exponent in the above formula. If the average correctness is 80%, then 1,000 components (a large program, remember?) at 80% gives a total correctness on the order of 10^-97, that is, damn-near-zero! Okay, maybe 1,000 components is too many - how about 100? If each component is only 80% correct, the total correctness is once again D.N.Z (10^-10). Ten components? Okay, you're around 10% correct overall!

Think about that for a second.

Now think about how many components you have on your current project. You can use the number of Java classes as a ballpark figure (if you're using Java like me). How correct is each class? Can you prove it?

If you followed Professor Dijkstra's logic, you'll recognize that you'd better have some serious unit tests for each component comprised in your program! Integration tests don't hurt either (but for reasons of combinatorial explosion, you can't thoroughly exercise your components using integration tests alone).

Corollary to the first part of this section:

Program testing can be used to show the presence of bugs, but never to show their absence! --Dijkstra / EWD249 (pp. 11)