JaVa
   

Organizing and Running Tests

Our mini-example has not placed major requirements on the test organization and test execution yet. The two test classes, DictionaryTest and DictionaryParserTest, can easily be restarted over and over again in the same test runner, and the execution time is negligible. However, as our project grows—and with it the number of test classes—this naive approach fails; we need to do some more thinking about test organization and test execution.

Organizing Tests

The first question is, Over how many test classes should I distribute my tests? So far, we packed all tests of an app class into a separate test class. As a rule of thumb, one test class per app class is normally suitable as a starting point, but there are good reasons to deviate from it now and then:

Another issue worth discussing is whether test classes belong to the same or a separate Java package and whether or not to separate test code from app code. Weighing up the benefits and drawbacks of the conceivable solutions does not lead to a clear judgement:

All three approaches have their specific benefits and drawbacks. The main thing is to use a common standard across the entire team. Further advantages can be achieved by agreeing on how to address all tests of a package, a subsystem, or the entire project. To this end, AllTests classes represent a quasi-standard. Typically, such a class exists for each package where the test classes are located. This looks as follows in our example:

import junit.framework.*;
public class AllTests {
 public static void main(String[] args) {
 junit.awtui.TestRunner.run(AllTests.class);
 }
 public static Test suite() {
 TestSuite suite = new TestSuite("Test suite for chapter3.*");
 suite.addTestSuite(DictionaryTest.class);
 suite.addTestSuite(DictionaryParserTest.class);
 return suite;
 }
}


While the suite() method groups all test classes of the package, the main() implementation offers us a comfortable point to start the test suite without having to worry about parameters. Which one of the three test runners is used for this purpose is again a matter of taste. The important thing is that, when adding, deleting, and renaming test classes, we must not forget to adapt AllTests accordingly. Depending on the system size and structure, at least one additional AllTests class exists, which groups all other AllTests suites into a test suite, for example:

package myproj.alltests;
import junit.framework.*;
class AllTests {
 public static void main(String[] args) {
 junit.awtui.TestRunner.run(AllTests.class);
 }
 public static Test suite() {
 TestSuite suite = new TestSuite("All tests of MyProject");
 suite.addTest(myproj.pack1.AllTests.suite());
 suite.addTest(myproj.pack2.AllTests.suite());
 suite.addTest(myproj.pack3.sub1.AllTests.suite());
 suite.addTest(myproj.pack3.sub2.AllTests.suite());
 return suite;
 }
}


The introduction of additional intermediate hierarchies for AllTests classes can simplify partial testing and updating of larger projects. Screenshot shows a conceivable package and class structure for a fictitious project. Here, myproj.pack3.alltests.AllTests groups all tests from the subpackages of myproj.pack3 together.

Package structure with test classes.

 

/myproj/pack1/
 ClassA.java
 ClassATest.java
 ClassB.java
 ClassBTest.java
 AllTests.java
/myproj/pack2/
 …
 AllTests.java
/myproj/pack3/subpack1/
 …
 AllTests.java
/myproj/pack3/subpack2/
 …
 AllTests.java
/myproj/pack3/alltests/
 AllTests.java
/myproj/alltests/
 AllTests.java

Screenshot: Package structure with test classes.

An alternative approach to the manual maintenance of test suites is to have all test classes of a package or even of an entire app generate and group automatically, by searching all subclasses of junit.framework.TestCase, for example. At first sight, this approach appears easier; the downside is that control over volume, organization, and distribution of all tests is easily lost.[10] One reviewer put it straight:

I can add nothing to this and prefer the manual maintenance of test suites, myself.

Running Tests

There are numerous opportunities to run a test suite [Gassmann00]:

In this respect, it is meaningful to select the number of executed tests so that as many test runs as possible are done, as long as this does not give a feeling that your work is hindered by unnecessary waiting. As long as a suite, including all tests of the system, runs in a few seconds, it won't hurt to start the full test set again whenever you make a local change. On the other hand, experience has shown that the runtime for all unit tests will be noticeable sooner or later. This is the reason we try to select a smaller execution granularity, depending on the individual case: subsystem suite, package suite, single test class, and sometimes even a single test case. The following is typical:

At the latest, you should test upon completion of a task and before the integration of modified classes; however, all tests of the system should be run. If you have a feeling that the runtime of the tests hinder you in running the tests regularly, then you could try to reduce the runtime, perhaps by using the techniques described in . [10]Java code to automatically find all TestCase subclasses and create a TestAll suite is found in an article by Schneider [00].


JaVa
Comments