Reworking Single Tests

Looking at a single test case, a testXXX method, we can see that several points deserve our attention:

• The test name should describe the functionality tested and perhaps the particular side conditions of that test. For example, a name like testAddUser is surely easier to understand than test1, and testAddUserWithoutPassword is better than testAddUserThrowException. The important thing is that it should be easy for readers of the test code to orient themselves and to identify the corresponding tests when changes are necessary. In particular, I do not recommend a naming convention that introduces consecutive numbers in any form. Numbering makes it more difficult to identify and maintain test cases.
• The length of a test method should be as short as possible. This can be achieved either by extracting method parts or by splitting the test. The finer the granularity of a single test, the easier it will be to understand the purpose of a test and to modify it when necessary. Having more than a handful of assertions should make you consider splitting the test.

  Testing complex scenarios and use cases often requires many individual steps. In such cases, we have to weigh carefully whether or not relocating parts of the test method to private methods could impair the readability of the test sequence.

• Test code contains as little logic as possible. Loops, branches, and case statements can be an indication that either the test is too complex or it tests more than a single unit. Auxiliary functions or auxiliary classes, to compare structured objects for instance, can represent exceptions to this rule and will require their own test cases.

  When testing legacy software logically, complex testing code is often necessary. This should be an intermediate state, though, and be changed as soon as the major refactorings have taken place.

• The expected results should be stated as previously determined constants and not be calculated in the test. The next example compares two variants of the same test to better explain this rule:

  public void testBalance1() {
   Account account = new account();
   account.deposit(10);
   account.withdraw(5);
   account.deposit(6);
   assertTrue(11, account.getBalance());
  }
  public void testBalance2() {
   Account account = new Account();
   account.deposit(10);
   account.withdraw(5);
   account.deposit(6);
   int balance = 10 - 5 + 6;
   assertTrue(balance, account.getBalance());
  }
  

  Following the example from testBalance2(), we can see that the test code implements almost all functionality of the app classes again. It is therefore meaningful to calculate the expected results in advance. An exception to this rule applies when the input data are already variable. In such a case, so-called test oracles (see Glossary) can help determine the correct result data to use in comparing the actual test results. ^[2] Note that oracles are used very seldomly in unit tests.

• Test data and expected results should be located close together. Consider this example:

  public void testIsPasswordValid() {
   assertTrue(user.isPasswordValid("abcdef"));
   assertFalse(user.isPasswordValid("123456"));
  }
  

  A reader of this code can see whether or not this test is correct only by finding the place where the User object is created. In contrast, the following test is easy to understand:

  public void testIsPasswordValid() {
   User user = new User("Name", "abcdef");
   assertTrue(user.isPasswordValid("abcdef"));
   assertFalse(user.isPasswordValid("123456"));
  }
  

  However, if we want to include the creation of User in the setup to avoid code duplication, then the use of constants offers a convenient middle course:

  public void testIsPasswordValid() {
   assertTrue(user.isPasswordValid(CORRECT_PASSWORD));
   assertFalse(user.isPasswordValid(WRONG_PASSWORD));
  }
  

The following rule applies in general: the bigger the distance between input and output data, the more difficult it will be to understand the test. For example, if we decide to relocate test data into a file, then the expected results should also be in this file. The next best solution would be to have a file with a similar name (e.g., testData.input and testData.expected) near the test itself.

• Exceptions that could potentially reach the test method and represent a test error should not be caught. ^[3] We therefore prefer:

  public void testRetrieveUser() throws WrongPasswordException {
   assertNotNull(manager.retrieveUser("name", "password"));
  }
  

  instead of the following:

  public void testRetrieveUser() {
   try {
   assertNotNull(manager.retrieveUser("name", "password"));
   } catch (WrongPasswordException e) {
   fail("Exception occurred: " + e.getMessage());
   e.printStackTrace();
   }
  }
  

  The try-catch code inflates the code without offering any real, informative gain when the exception or its message string are not specific enough. This negative example is still useful, because it demonstrates a JUnit command that we haven't covered yet: "fail(String text)." This command is also available in the TestCase class and always triggers a failure, which is meaningful, for example, when the correctness of a test case is determined by the program flow. Usually we don't even declare the specific exception type(s) in the test method's throws clause but instead use the generic Exception or Throwable:

  public void testRetrieveUser() throws Exception {
   assertNotNull(manager.retrieveUser("name", "password"));
  }
  

  The various possible exception types are of no interest to the invoking test method and the generic approach greatly reduces maintenance.

Each team will develop their own basic vocabulary with test idioms and guidelines during the course of a project. Things that win through standardization include the use of the optional string parameter in assert calls or the naming of test fixture variables. One of the important aspects of guidelines is their consistent use. Far less important are the concrete contents of individual rules. ^[2]Binder [99, p. 917] dedicates an entire chapter to oracles. ^[3]An excellent example of how greatly opinions can vary is demonstrated in Gassmann [00], where the opposite rule is stated.

• Previous
• Next