Well, isn't that typical! you will probably say. The author gives fishy, commonplace answers to the most important questions. I'd like to know what I should test and what I shouldn't! To contain your temper a little, I will try to give you a few "concrete" answers in the following sections.

Tests per Class

Should each class have its own test class?

Yes. Each nontrivial class should have at least one test class. Trivial classes are, for example, exception classes that do nothing but implement standard constructors and pass parameters to super. The differentiating answer is, Does that class have its own logic?

Getters and Setters

Should I test simple getters and setters of attributes?

Ron Jeffries says no, because we can see at a glance that they are correct. I don't accept that as a sufficient argument because we cannot be sure that extensions and refactorings will have an impact on the code which we currently know to be correct. However, most of the time these trivial methods don't require dedicated tests since they are already used-and thereby tested-in other test cases. If this is not the case, then a dedicated settergetter test method will at least do no harm.

Non-Public Object Properties

A brief definition of terms appears to be useful at this point. We use the term public for everything that can be accessed from outside of our unit under test. When testing a single class, then protected and package-scope methods are also regarded as public. In contrast, when testing a subsystem where all classes are within one package, then only methods and constants with the access specification public are public. This leads us to two related, but different questions:

Should we test non-public methods? []

The supporters argue that even private methods can be very complex so that something in them can go wrong. However, the drawbacks in testing private methods predominate. They make refactoring more difficult and increase the maintenance effort, because they are strongly implementation-dependent tests, and we have to use a few tricks to be able to invoke them at all (as we next discuss). When using the test-first approach, wishing to test private methods is mainly a matter. Either the method in question is actually part of the public interface (for which there is no client code yet) and if so, we should simply replace private with public and everything will be alright. Or, a new helper object is waiting to be born and in that event the method would be public. However, everything will look different during subsequent testing. Unfortunately, there is rarely an alternative to grasping the nettle: before we can start cleaning up inherited unknown code, we need workable tests as a safety net (see also , , Unit Testing for Existing Software. In such a case, we can only hope that these tests will prove to be a mere transitional stage.

Should we allow tests to access internal things?

By "internal things," we mean non-public methods and attributes. The benefit is that it would allow us to test many post-conditions with much less effort, compared to the situation where we have to rely on the public interface. The drawback is again the high implementation dependence of such test cases. Ron Jeffries answers this question pragmatically: "those who believe that an important test can be realized only by accessing implementation details should do this. If the implementation proves to be too erratic, we will notice it very quickly. On the other hand, if the implementation remains stable, then the non-public feature may be an unrecognized public feature looking for its client." [] Once we have decided to break the object's private sphere, we will have to clarify how to technically access private methods and attributes. The following options are available: we can change the access restriction for testing purposes, or we can use reflection (from JDK 1.2 up) or mock objects to mitigate the problem and possibly avoid it.

What is the bottom line? Try to avoid testing private methods and accessing internal properties of a class within your tests. If you change your mind after careful consideration, then this won't mean the end of the world in most cases.

Complex Interaction Tests

We explained in , , why interaction tests between two objects are sometimes necessary to create an adequate test suite. However, if we do unit testing without dummy or mock objects, we will soon get to a point where we have to create a complex object mesh to be able to create a test fixture, and where the distance between the OUT and the basic objects of the fixture becomes large. Such tests are difficult to read and maintain, and they also violate the rules requiring maximum independence and minimum granularity for our tests. The benefit of such micro integration tests is that they can uncover "random" errors by indirect use of numerous other objects. The successful tester lives on such planned chances. So when should we avoid this type of complex test? The following comments can give us useful indications:

  1. The test runs are very long, because they access databases or other external resources.
  2. We can no longer adapt our tests after changes to the code or when new requirements emerge, because there are excessive cross-dependencies.
  3. The tests often raise false alarm, because things in the remote object have changed, while these things are not relevant for the actual test case.
  4. The tests overlap functionally with the acceptance tests.
  5. The tests use real data-production data dumps provided by the customer-which are too complex for us developers to remember all the details.
  6. Understanding the test scenarios requires extensive customer-specific knowledge that the average developer does not typically have.

Points 1 through 3 occur mostly when the code under test was not at all or only partly developed by the test-first approach, because test-first code is generally less dependent than design-first code. If we have a feeling that we cannot test our CUT in isolation or without a complex fixture, then this clearly indicates that there is a design problem that will have to be solved sooner or later.

Points 4 through 6 indicate that we have exceeded our developer competence and entered the realm of acceptance tests. This may be necessary when customers do not supply their own acceptance tests. But in these cases, too, we should draw an organizational separation between the "real" unit tests and the developer's own acceptance tests, by using separate packages, for example.

Testing the Tests

One frequently asked question is whether or not we should write tests for the tests themselves. The simple answer is no, because test cases do not contain logic that has to be verified. If they do contain logic (e.g., in the form of branches) then that test case has to be simplified (see also , ). According to our experience a trivial error in the test code, for instance, a wrong assertEquals(...), will be discovered during the first test run in most cases. From this perspective, the app code is sort of a test for the test code.

One exception to the rule, No tests for the sake of tests, is testing helper classes with their own logics (e.g., reusable dummy objects). They actually need their own test suite. []This question is one of the most fiercely discussed issues in the field of unit tests under XP (see ). []One question closely related to this issue is whether data encapsulation is generally overvalued, such that all methods would better be public. Some argue that the Smalltalk world copes wonderfully with the fact that "private" represents merely a guideline and is not forced by the language. You can read the discussion and get involved at .