Ever since annotations officially made it officially in the Java
language, the debate has been raging about where configuration should
go:  in annotations or XML files.

The pros and cons of each approach are easy to summarize:
 annotations allow you to put your configuration system close
to the Java source it applies to, but changing it requires Java
knowledge and a recompilation.  On the other hand, XML files
are usually easy to modify and they can be reread at runtime (sometimes
even without relaunching your application), but they are very verbose
and the edition can be error prone.

As usual in such situations, the correct course of action is to use a
mix of these approaches.  For example, a rule of thumb that I
have been following is that “any configuration parameter that refers to
a Java element (package, class, method) should be specified in an
annotation” (e.g. @Test or @Transaction)
and “any configuration
parameter that applies to your application as a whole probably belongs
to an XML file” (e.g the name of an output log file or a target web

Another approach that blends these two techniques has also been under
scrutiny recently:  being able to override Java annotations
with XML files.  I haven’t come across an implementation of
this idea that can claim it’s the defacto standard, and at this point,
I suspect that this approach is unlikely to take off because it suffers
from the “worst of both worlds” syndrome.

Here is why.

Consider the simple following code:

public class Mytest {
  @Test(invocationCount = 10)
    public void verify() {
    // ...

This instructs TestNG to invoke the verify() test
method ten times.

If I want to override this value at runtime, I find myself having to
write a fairly complex piece of XML code in order to pinpoint the exact
annotation I’m trying to override.  It would probably look
like this:

  <package name="org.foo.tests">
    <class name="MyClass">
      <method name="verify">
        <annotation name="org.testng.Test">
          <attribute name="invocationCount" value="15" />

Of course, you might want to come up with a way to capture the override
(the annotation/attribute part) so you can reuse it somewhere else in
your XML file, in case you want this override to apply to more than
just one method.  This would probably be achieved with the
definition of an “override-ref” that you would define at the top of
your XML file and that you would repeatedly use throughout your XML
file (exactly like ant’s classpath-ref, for example).

That’s already a lot of work (and it’s quite hard to read), but it’s
also extremely fragile since it will most likely break if you decide to
rename your class or your method name (IDE’s are beginning to extend
refactorings to non-Java files, but we’re not quite there yet).

Because of these shortcomings, I have been working on a slightly
different approach with
TestNG that lets you specify this runtime override in Java.
 Again, it’s not a silver bullet and suffers from some of the
compromises expressed above, but if you can live with the idea that you
have to recompile your override if you modify it (but not the annotated
code you are trying to override, so it’s already a progress), it’s
actually fairly simple to achieve.

TestNG introduces the following interface:

public interface IAnnotationTransformer {
  public void transform(ITest annotation, Class testClass,
                        Constructor testConstructor, Method testMethod);

and you use it as follows:

public class MyTransformer implements IAnnotationTransformer {
  public void transform(ITest annotation, Class testClass,
                        Constructor testConstructor, Method testMethod)
    if ("verify".equals(testMethod.getName())) {

Here are a few observations about this technique:

  • The idea is that whenever TestNG parses a @Test
    it will run it through the annotation transformer before using it.

  • The three extra parameters of the method let you know where
    this annotation was found (on a class, a constructor or a method).
     Only one of these three parameters will be non-null.

  • Notice that the parameter to the transform()
    method is an
    ITest.  This is because TestNG
    supports both
    standard and JavaDoc annotations, so ITest is a
    façade object that represents the @Test
    (JDK) or
    @testng.test (JavaDoc) annotation.
     In a more general
    framework, the parameter would probably directly be the type of the
    annotation, although there are still a few problems to solve with this
    (see the next point).

  • This approach requires write access to the annotation,
    which is not possible with the current JDK implementation.
     You can work around this limitation by using a
    façade, which could possibly be generated as a dynamic proxy.

The advantage of annotation transformers is that they can be very
powerful:  it’s very easy to apply them to several methods,
several classes, all methods of a class or of a package, etc…

Annotation Transformers are alreay implemented in the TestNG Subversion
, and they will be available publicly when TestNG 5.3 comes out
(very soon).