TestNG Makes Java Unit Testing a Breeze

by: Filippo Diotalevi

Try this testing framework for its advances over JUnit
The JUnit framework is the current the one-stop solution for Java language unit tests. This framework deserves praise for introducing the idea of test-driven development to Java developers and teaching them how to effectively write unit tests. However, JUnit has only marginally evolved during the last few years; thus, writing tests for today's complex environment has become an increasingly difficult task, in which JUnit must be integrated with several other complementary test frameworks. In this article, Filippo Diotalevi introduces TestNG, a new framework for testing Java applications. TestNG isn't just really powerful, innovative, extensible, and flexible; it also illustrates an interesting application of Java Annotations, a great new feature in JDK 5.0.
In the construction phase of every modern software package, the practice of testing plays a central role. Gone are the days of coding first and testing whenever you have time (or not at all), as most developers now recognize the need to adopt a software methodology in which coding and testing are intermingled and contemporaneous, so as to catch bugs early and identify major risks at the very beginning of the development process.

More than any other testing framework, JUnit has driven developers to understand the usefulness of tests, especially of unit tests. Leveraging a rather simple, pragmatic, and strict architecture, JUnit has been able to "infect" great number of developers. (See Resources for more information on being "test infected.") JUnit users have learned some fundamental rules of unit testing:

• Every single piece of code must be tested.
• Whenever possible, code must be tested in isolation (using, for example, techniques like mock objects).
• Software must be easy testable -- that is, written with tests in mind.

However, as developers' confidence with tests has increased, JUnit's simplicity and strictness has divided them into two opposed factions. On the one hand, there are those who firmly believe that JUnit's simplicity is necessary to continuously remind programmers that software must be simple too (this is known as the KISS principle, for keep it simple, stupid); on the other hand, there are those who, seeing JUnit more as simplistic than simple, want new advanced features, more flexibility, and more power from their testing framework. Some peculiar features of JUnit come in for particular criticism from the latter group:

• The need to extend a TestCase class, because the Java language has single inheritance, is very limiting.
• It is impossible to pass parameters to JUnit's test method as well as to setUp() and tearDown() methods.
• The execution model is a bit strange: The test class is reinstantiated every time a test method is executed.
• The management of different suites of tests in complex projects can be very tricky.

The creator of TestNG
The creator of TestNG is Cedric Beust, a well-known name in the field of Java programming, a member of the EJB 3 expert group, and creator of other popular open source projects like EJBGen and Doclipse. TestNG is distributed under the terms of Apache Software License and can be downloaded from its Web site (see Resources for a link to this and to Cedric's site).

In this article, you will learn how to write unit tests for an application using a new testing framework called TestNG. TestNG takes its inspiration from JUnit, trying to maintain the latter's simplicity; at the same time, however, TestNG eliminates most of the limitations of the older framework and gives the developer the ability to write more flexible and powerful tests. As it heavily borrows from Java Annotations (introduced with JDK 5.0; see Resources for more information on this new feature) to define tests, it can also show you how to use this new feature of the Java language in a real production environment.

About the code
To illustrate the use of TestNG, I'll write some unit tests for a widely used open source library called Jakarta Common Lang, which contains some useful classes to handle and manipulate strings, numbers, and Java objects. In the Resources section below, you'll find links both to TestNG and the Jakarta Common Lang libraries; you'll need to download both if you plan on following along with this article on your own machine.

TestNG is available in two different packages: one that requires JDK 5.0 and another one that is compatible with version 1.4 of the language. They use a slightly different syntax to define tests: The former uses JDK 5.0 annotations, while the latter uses old Javadoc-style annotations. This article uses the JDK 5.0 version, so you'll need to have a basic understanding of annotations before you continue with this article; you can find links to developerWorks resources on the topics in Resources. However, it is important to be aware that JDK 5.0 is required only to compile and run the tests; you can still build your application code with your preferred compiler. In fact, you will test the Jakarta Common Lang library using the same JAR file that you can download from the Jakarta project's Web site. You can find more details about using TestNG with version 1.4 of the Java platform from TestNG's Web site.

Finally, click the Code icon at the top or bottom of this article to download j-testng-sample.zip, which contains some examples illustrating how to use TestNG to write unit tests for Jakarta Commons Lang. You will find most of the code presented here, along with some other samples. You won't need this code to read the article, but it can help you understand the concepts explored here in more depth.

TestNG quickstart
A TestNG test class is a plain old Java object; you don't need to extend any special class or use any naming convention for test methods: You just use the annotation @Test to signal to the framework the methods of the class that are tests. Listing 1 illustrates one of the simplest tests possible for the utility class StringUtils. It tests two methods of StringUtils: isEmpty(), which checks a String for emptiness, and trim(), which removes control characters from both ends of a String. Note that the assert Java instruction is used to check error conditions.

Listing 1. A test case for class StringUtils

package tests;



import com.beust.testng.annotations.*;

import org.apache.commons.lang.StringUtils;



public class StringUtilsTest

{

	@Test

	public void isEmpty()

	{

		assert StringUtils.isBlank(null);

		assert StringUtils.isBlank("");

	}



	@Test

	public void trim()

	{

		assert "foo".equals(StringUtils.trim("  foo   "));

	}

}

Before you can run the tests, however, you must configure TestNG using a special XML file, conventionally named testng.xml. The syntax for this file is very simple, and is presented in Listing 2. This file begins by defining a test suite, My test suite, composed of a unique test, First test, that is made by the StringUtilsTest class.

Listing 2. Configuration file for TestNG

<!DOCTYPE suite SYSTEM "http://beust.com/testng/testng-1.0.dtd" >

<suite name="My test suite">

  <test name="First test">

    <classes>

       <class name="tests.StringUtilsTest" />

    </classes>

  </test>

</suite>

If this sample testng.xml file doesn't seem very useful (there's only one test class), the great news is that it is actually the only file you need to write to define your test suites. Remember the old days of JUnit? In those days, the definition of your suites was probably spread over several files: JUnit's TestSuites, property files, and, obviously, Ant build files. With TestNG, all the data you need is gathered in the testng.xml file. No more TestSuites, and a thinner build file.

To run the test, you compile the class with javac and then invoke TestNG with the following command:

java -ea -classpath .;testng.jar;commons-lang-2.0.jar com.beust.testng.TestNG testng.xml

Here, the option -ea tells the JVM to handle assertions (and to raise an exception when an assertion fails); testng.jar and commons-lang-2.0.jar are the only two libraries required to run this example, and com.beust.testng.TestNG is the main class of TestNG. For all you lazy developers who have happily forgotten the cryptic syntax of java and javac, a useful Ant task is also available. Listing 3 presents, as an example, the Ant build file of the sample application distributed with this article. Note the definition of the testng task associated with the class com.beust.testng.TestNGAntTask, and its rather simple usage in the test target.

Listing 3. Ant build file with TestNG task

<project name="sample" default="test" basedir=".">



   <!-- COMPILE TESTS-->

   <path id="cpath">

      <pathelement location="testng.jar"/>

      <pathelement location="commons-lang-2.0.jar"/>

   </path>

   <target name="compile">

      <echo message="compiling tests"/>

      <mkdir dir="classes"/>

      <javac   debug="true"

         source="1.5" classpathref="cpath"

         srcdir="src" destdir="classes"/>

   </target>



   <!-- RUN TESTS-->

   <taskdef name="testng"

      classname="com.beust.testng.TestNGAntTask"

      classpathref="cpath"/>

   <path id="runpath">

      <path refid="cpath"/>

      <pathelement location="classes"/>

   </path>

   <target name="test" depends="compile">

      <echo message="running tests"/>

      <testng fork="yes" classpathref="runpath" outputDir="test-output">

         <fileset dir="src" includes="testng.xml"/>

         <jvmarg value="-ea" />

      </testng>

   </target>



</project>

If all has been done correctly, you should see the results of your tests in the console. Furthermore, TestNG creates a very nice HTML report in a folder called test-output that is automatically created in the current directory. If you open it and load index.html, you will see a page similar to the one in Figure 1.

Figure 1. HTML report created by TestNG

Defining test groups
Another interesting feature of TestNG is its ability to define groups of tests. Every test method can be associated with one or more groups; once you've defined these groups, you can choose to run only certain groups of tests. To add a test to a group of tests, simply specify the group as a parameter of the @Test annotation, using the following syntax:

@Test(groups = {"tests.string"})

In this particular case, you declare that the annotated method belongs to the tests.string group. Because the parameter groups is an array, it is possible to specify multiple groups, separating their names with commas. For example, in the sample application, you can create different tests for Strings, numbers, and booleans, and then run them selectively, configuring TestNG as illustrated in Listing 4.
Listing 4. Configuration file with different groups

<!DOCTYPE suite SYSTEM "http://beust.com/testng/testng-1.0.dtd" >

<suite name="My suite">

  <test name="Simple example">

    <groups>

      <run>

		<include name="tests.string" />

		<include name="tests.math" />

		<exclude name="tests.boolean"/>

      </run>

    </groups>

    <classes>

		.... list classes here....

    </classes>

  </test>

</suite>

Obviously, when you run different groups of tests, the HTML report can show all the tests in a single list, as well as in separate lists for each group, making it possible to immediately understand the source of any problem.

Configuration methods
With TestNG, you can specify more than just test methods; it's also possible to specify other particular methods in a class, called configuration methods, using the dedicated annotation @Configuration. There are four types of configuration methods:

• beforeTestClass methods are executed after the instantiation of the class but before any test method has been run
• afterTestClass methods are executed after every test method in a class has been run
• beforeTestMethod methods are executed before the execution of any test method in a class
• afterTestMethod methods are executed after the execution of every test method in a class

The lifecycle of a test class is further illustrated in Figure 2.

Figure 2. Lifecycle of a test class

Listing 5 illustrates some examples of configuration methods. Please note that if you use groups, configuration methods must also belong to a group. Furthermore, the four types of configuration methods are not mutually exclusive, so it is possible to define methods that belong to one or more of these categories at the same time (see the aroundTestMethods() method in Listing 5 for an example).

Listing 5. Examples of configuration methods

 @Configuration(beforeTestClass = true, groups = {"tests.workflow"})

public void setUp()

{

   System.out.println("Initializing...");

}



@Configuration(afterTestMethod = true, beforeTestMethod = true, groups = {"tests.workflow"})

public void aroundTestMethods()

{



   System.out.println("Around Test");

}

Configuration methods in TestNG are a more powerful version of JUnit's setUp() and tearDown() methods; their main purpose is to create the right execution context for a test and to refresh data after the execution of a test case.

Exception checking
With TestNG, you can check the occurrence of an exception very simply and easily. It is obviously possible to do this with JUnit as well, but using the @ExpectedExceptions annotation with TestNG makes the code of the test surprisingly easy and straightforward to write, as you can see in the example presented in Listing 6. The @ExpectedExceptions annotation specifies that the raising of a NumberFormatException is tolerated by the framework and therefore should not be considered a failure. To see if an exception is raised in a certain line of code, you can add an assert false statement just after that line. This means that you will pass the test only if that particular type of exception is raised in the designated line.

Listing 6. Exception checking with TestNG

public class  NumberUtilsTest

{

	@Test(groups = {"tests.math"})

	@ExpectedExceptions(NumberFormatException.class)

	public void test()

	{



		NumberUtils.createDouble("12.23.45");

		assert false;  //shouldn't be invoked

	}



}

Wrapping up
In this article, I've offered a quick and practical introduction to TestNG, with the aim of showing how you can start writing unit tests now. However, it isn't a complete reference manual. There are a lot of other interesting features of TestNG that can be very useful:

• It's possible to pass arguments to test and configuration methods, declaring them either with annotations or in the XML configuration file.

• You can run good old JUnit tests under TestNG using a "compatibility mode."

• It's possible to establish dependencies among test groups, deciding their order of execution.

A look at TestNG's documentation (see Resources) is necessary to exploit all the potentialities of this framework.

All these features, together with the adoption of Java annotations to define tests, make the whole testing process much more simple and flexible. There are only a few rules that you must obey to write tests; beyond these, you are absolutely free to choose the testing strategy you prefer.

What clearly emerges while using TestNG is that this framework is already a good choice for writing unit tests, and that it is, by design, simple to integrate with other libraries and tools, so its future development can bring developers some interesting news. A new path has been forged.

Resources
Click the Code icon at the top or bottom of this article to download j-testng-sample.zip, the code discussed in this article.

Download TestNG at the project's Web site.

• You'll need to download the Jakarta Commons Lang library to run this article's code.

• If you're unfamiliar with Java annotations, check out Brett McLaughlin's two-part developerWorks series on the subject from September 2004:
Part 1: Add metadata to Java codePart 2: Custom annotations

• You can download JUnit at JUnit.org

• "Incremental development with Ant and JUnit," Malcolm Davis (developerWorks, November 2000) is a good article explaining how JUnit can be integrated into your projects.

• Eric Allen and Roy Miller both covered unit testing frequently in their respective columns, Diagnosing Java code and Demystifying Extreme Programming.

• Testdriven.com is a comprehensive collection of articles and resources about test-driven development.

• Learn more about various open source testing tools.

• Learn more about "Unit testing with mock objects," Alexander Day Chaffee and William Pietri (developerWorks, November 2002).

• A lot of ideas about JUnit, unit testing, and TestNG are recorded on Cedric Beust's blog.

• Are you test infected? Find out the meaning of this term.

• Visit the Developer Bookstore for a comprehensive listing of technical books, including hundreds of Java-related titles.

Download

 

 Name    Size    Download method   

 j-testng-sample.zip        FTP