What Can Testing Achieve?

It's impossible for testing to guarantee that a program is correct. However, testing can provide a high level of confidence that a program does what we expect of it. Often "bugs" reflect ignorance about what code should really do. As our knowledge of what a program should do grows, we can write tests that tighten its requirements. It is important to recognize the limitations of testing – testing won't always expose concurrency issues. Here, an ounce of prevention is truly worth a pound of cure (for example, testing may well fail to pick up problems relating to instance data in a servlet being modified concurrently by multiple threads). However, such code will surely fail in production, and no competent J2EE developer should write it in the first place.


The longer a bug takes to appear, the more costly it will be. One study found that the cost of eventually fixing a bug multiplied by 10 with each phase of a project – requirements, design, implementation, and post-release – that passed before the bug was spotted. Testing is no substitute for careful thought before writing code; testing can never catch all bugs.


While in this chapter we'll focus on testing code, it's important to remember that a sound QA strategy is needed from requirements analysis onwards.