Custom CSS

Friday, November 17, 2006

Using Cobertura for code coverage reporting

Cobertura is a coverage tool for Java that integrates nicely with Ant. The output is pretty and simple to digest, plus it is easy to setup.

From the Cobertura website:

Why use a coverage tool?

No matter how good [one's] methodologies are, and how diligently they're followed in an organisation, it isn't possible to ensure that software testing is as comprehensive as it could be. That's where tools can help.

Cobertura is a free, simple and easy-to-use tool that will complement your existing Java development practices. It helps you discover exactly where your software is being tested and, more importantly, where it isn't. Cobertura will help you to view your software from a number of levels, from the entire system right down to an individual line of code.

In a nutshell, Cobertura will tell you exactly which lines of code were executed after running your unit tests, what percentage of the class was covered by unit tests, and how complex your code is. However, there are still some gotchas for getting Cobertura working with Ant that some online tutorials don't tell you, and that is the subject of today's article.

Cobertura requires 3 libraries in its classpath:

I have set up a special classpath that I use for instrumenting classes in Ant. Note that the aforementioned libraries are the [i">only[/i"> ones required for instrumentation, so those are the only ones in my classpath. The reason for this is that other tools such as Hibernate may use other incompatible versions of ASM, and it may cause your instrumentation endeavor to fail.

<path id="instrument.classpath">
  <fileset dir="lib">
    <include name="cobertura.jar"/>
    <include name="jakarta-oro*.jar"/>
    <include name="asm-all-2.2.2.jar"/>
    <include name="log4j*.jar" />

Then I set up a task to instrument my compiled classes, and copy them back over to my compiled classes directory. This way, any of my other tasks can use the instrumented classes without even knowing that they've been instrumented. None of my other tasks change.

<target name="instrument" depends="init, compile">
    <taskdef classpathref="instrument.classpath"
    <cobertura-instrument todir="build/instrumented">
      <classpath refid="instrument.classpath"/>
      <fileset dir="build/classes">
        <include name="**/*.class"/>
    <copy todir="build/classes">
      <fileset dir="build/instrumented"/>

Note that after running this task, Cobertura creates a cobertura.ser file in your project root. This file contains information about the instrumented classes, and is written to during instrumented runs of your code.

Once you have run your unit tests (using whatever you like, however you like; your testing framework will never know your .class files have been instrumented), you can run a coverage report to have Cobertura print out a similar looking and pretty sample report linked above.

Here is my target that does the coverage report:

<target name="coverage-report" depends="init">
    <cobertura-report srcdir="src" destdir="doc/coverage-report">
    <delete failonerror="false">
      <fileset file="cobertura.ser"/>

Click here to see a sample report.