Archive for October, 2006

Announcing TestNG 5.3

I’m pleased to announce the immediate availability
of TestNG 5.3
.

The main feature in this new release is the introduction of Annotation
Transformers, that I described in a previous
post
.  The full documentation for Annotation
Transformers can be found here.
 As you can see below, the change log is quite big as well, so
if we failed to fix a bug you reported or an improvement you requested,
please let us know and we’ll try to get it in for the next release!

Thanks to all who helped make 5.3 a reality!

Core

  • Fixed: use a single instance of bsh.Interpreter
  • Added: @Before/@AfterMethod can declare a
    java.lang.reflect.Method parameter to be informed about the @Test method
  • Fixed: super classes must not be listed in
    testng-failures.xml
  • Fixed: parallel attribute must not appear if empty or null
    in testng-failures.xml
  • Fixed: parsing for javadoc annotations is done on request
    only
  • Added: improved multiple suite summary page report
  • Added: -target option deprecated in favor of -annotations
    javadoc|jdk
  • Fixed: filesets in the ant task didn’t work if the paths
    have spaces in them
  • Fixed: Before/After Suite were behaving wrong in parallel
    execution
  • Added: A generic/extensible RemoteTestNG was added to the
    core
  • Fixed: Before/AfterGroup-s were behaving wrong when using
    invocationCount, dataProvider and threadPoolSize
  • Fixed: improved support for running different annotation
    type tests in the same suite
  • Fixed: testng-failed.xml was generated even if there were
    no failures/skipps
  • Fixed: -usedefaultlisteners was wrongly passed to JVM
    instead of TestNG options
  • Added: Attribute dataProviderClass for @Test and
    @testng.test
  • Fixed: Forgot to account for cases where both
    invocationCount and DataProviders are present
  • Fixed: AfterGroups were invoked out of order with
    invocationCount and DataProviders
  • Fixed: Reporter.getOutput() returned an empty array if a
    timeOut was specified
  • Added: testng.xml now supports <suite-files>
  • Added: ant task can receive several listeners
  • Fixed: TESTNG-109 Skipped tests with expected exceptions
    are reported as failures
  • Added: ant task can now select the parallel mode for
    running tests
  • Fixed: ant task correctly deals with empty groups and
    excludedgroups parameters
  • Added: ant task can override default suite and test names
  • Added: comand line support for setting parallel mode, suite
    and test names

Eclipse plug-in

  • Added: Support for configuring per project
    usedefaultlisteners
  • Added: Contextual drop-down menu on failures tab of the
    TestNG view to enable running/debugging method failure only
  • Added: Suppport for configuring per project TestNG jar
    usage (project provided one or plugin provided one)

A view from below…





Click on the image for full size (2 megs)

Just when I thought I’d seen everything in World of Warcraft…

An underground subway runs between the two major Alliance cities
Stormwind and Ironforge.  For a long time, I noticed that this
Tram goes through an underwater section about halfway between the two
cities, but I never bothered to find out. It always intrigued me, though, so a few days ago, I took the long walk
to see for myself, and I wasn’t disappointed…

Through the
waving reflections of the force field that keeps the water from
flooding the tunnel, you can see a cute Deep Run Diver walking on the
sandy bottom, a sunken pirate ship and a mermaid.  But the
best is on the other side of the track…

And as for all World of Warcraft screenshots, I strongly recommend to
click on both of these pictures, they are worth it…

Annotation transformers in Java

Ever since annotations officially made it officially in the Java
language, the debate has been raging about where configuration should
go:  in annotations or XML files.

The pros and cons of each approach are easy to summarize:
 annotations allow you to put your configuration system close
to the Java source it applies to, but changing it requires Java
knowledge and a recompilation.  On the other hand, XML files
are usually easy to modify and they can be reread at runtime (sometimes
even without relaunching your application), but they are very verbose
and the edition can be error prone.

As usual in such situations, the correct course of action is to use a
mix of these approaches.  For example, a rule of thumb that I
have been following is that “any configuration parameter that refers to
a Java element (package, class, method) should be specified in an
annotation” (e.g. @Test or @Transaction)
and “any configuration
parameter that applies to your application as a whole probably belongs
to an XML file” (e.g the name of an output log file or a target web
server).

Another approach that blends these two techniques has also been under
scrutiny recently:  being able to override Java annotations
with XML files.  I haven’t come across an implementation of
this idea that can claim it’s the defacto standard, and at this point,
I suspect that this approach is unlikely to take off because it suffers
from the “worst of both worlds” syndrome.

Here is why.

Consider the simple following code:

public class Mytest {
  @Test(invocationCount = 10)
    public void verify() {
    // ...
  }}

This instructs TestNG to invoke the verify() test
method ten times.

If I want to override this value at runtime, I find myself having to
write a fairly complex piece of XML code in order to pinpoint the exact
annotation I’m trying to override.  It would probably look
like this:

<override-annotation>
  <package name="org.foo.tests">
    <class name="MyClass">
      <method name="verify">
        <annotation name="org.testng.Test">
          <attribute name="invocationCount" value="15" />
        </annotation>
      </method>
    </class>
  </package>
</override-annotation>

Of course, you might want to come up with a way to capture the override
(the annotation/attribute part) so you can reuse it somewhere else in
your XML file, in case you want this override to apply to more than
just one method.  This would probably be achieved with the
definition of an “override-ref” that you would define at the top of
your XML file and that you would repeatedly use throughout your XML
file (exactly like ant’s classpath-ref, for example).

That’s already a lot of work (and it’s quite hard to read), but it’s
also extremely fragile since it will most likely break if you decide to
rename your class or your method name (IDE’s are beginning to extend
refactorings to non-Java files, but we’re not quite there yet).

Because of these shortcomings, I have been working on a slightly
different approach with
TestNG that lets you specify this runtime override in Java.
 Again, it’s not a silver bullet and suffers from some of the
compromises expressed above, but if you can live with the idea that you
have to recompile your override if you modify it (but not the annotated
code you are trying to override, so it’s already a progress), it’s
actually fairly simple to achieve.

TestNG introduces the following interface:

public interface IAnnotationTransformer {
  public void transform(ITest annotation, Class testClass,
                        Constructor testConstructor, Method testMethod);
}

and you use it as follows:

public class MyTransformer implements IAnnotationTransformer {
  public void transform(ITest annotation, Class testClass,
                        Constructor testConstructor, Method testMethod)
  {
    if ("verify".equals(testMethod.getName())) {
      annotation.setInvocationCount(15);
    }
  }
}

Here are a few observations about this technique:

  • The idea is that whenever TestNG parses a @Test
    annotation,
    it will run it through the annotation transformer before using it.

  • The three extra parameters of the method let you know where
    this annotation was found (on a class, a constructor or a method).
     Only one of these three parameters will be non-null.

  • Notice that the parameter to the transform()
    method is an
    ITest.  This is because TestNG
    supports both
    standard and JavaDoc annotations, so ITest is a
    simple
    façade object that represents the @Test
    (JDK) or
    @testng.test (JavaDoc) annotation.
     In a more general
    framework, the parameter would probably directly be the type of the
    annotation, although there are still a few problems to solve with this
    (see the next point).

  • This approach requires write access to the annotation,
    which is not possible with the current JDK implementation.
     You can work around this limitation by using a
    façade, which could possibly be generated as a dynamic proxy.

The advantage of annotation transformers is that they can be very
powerful:  it’s very easy to apply them to several methods,
several classes, all methods of a class or of a package, etc…

Annotation Transformers are alreay implemented in the TestNG Subversion
depot
, and they will be available publicly when TestNG 5.3 comes out
(very soon).

FireFox 2: Please fix this before it’s too late

FireFox 2.0 just relased its Release Candidate 2, which means it’s nearing its final version. Overall, it’s much faster than 1.5 (the only new feature that I really care about, actually), but it is plagued by one change that is irking me to no ends: they changed the tabbed UI.

Compare the 1.5 version:

to the 2.0 version:

With FireFox 1.5, there is only one close button (at the very end of the bar) and all the tabs open are visible at all times. With FireFox 2.0, each tab has an individual close button and instead of bunching up, the open tabs continue off the screen, and a drop-down allows you to jump to one that’s not currently visible.

This is very wrong for two reasons:

  • The close button now changes places all the time. I just never know where
    to find it:  I need to first find what tab is currently highlighted (not very easy to do with MacOS because the difference between the active and inactive tabs is subtle) and then click it. After two weeks using FireFox 2, I still find myself closing the wrong tab once in a while. Actually, I’ve sometimes even closed the tab I wanted to jump to by accident because I clicked on its close button when I only intended to activate it…

  • More than once now, my muscle memory have led my mouse cursor toward the
    rightmost close button, thereby expecting to close the tab I’m currently
    reading while, of course, closing one I haven’t read yet.

This is such a huge UI mistake that I’m still shaking my head in disbelief.

Here is the rationalization offered on the Mozilla Web site:

Power users who open more tabs than can fit in a single window will see arrows on the left and right side of the tab strip that let them scroll back and forth between their tabs.

Well, sorry, but that doesn’t hold water, and I’m sure that if they had conducted even only one usability study, they would have found that there is a vast majority of users who just think the older UI is more functional.

But I’m open-minded, and I certainly understand the value of evolving user interfaces over time, but when you are altering something that is so fundamental to basic navigation, the least you can do is offer a switch to users who prefer the older behavior (like Eclipse wisely did). Unfortunately, such a setting is nowhere to be found in the Tabs configuration.

I hope the FireFox developers do the right thing and introduce this setting before 2.0 goes final.

TestNG is getting even more Groovy

The indefatigable Andy Glover strikes again with his copacetic post
"TestNG
is so Groovy"
.

We all know that Groovy compiles to standard .class files, and in this post,
Andy comes up with a technique that, from now on, will let him write his tests
in any combination of

  • TestNG / JUnit and
  • Java / Groovy

Write once, test anywhere!

 

Mac OS background madness

I like simple, uncluttered desktops, and I realized a long time ago that I
was among a very small group of users for doing so.  Most of the computer background
desktops that I see are usually filled with documents, links and programs to a
point that I wonder how these users ever find what they’re looking for. 

Well, I’m sure they find their documents eventually, but do they really visually search
through the rows of 10×10 icons on their desktop to do so, or do they rely on
the Explorer or Google Desktop for this?  On top of this, most users
typically have the picture of a
sunset, a waterfall or their sons playing soccer, further degrading the
readability of whatever text they used to name their documents.

If you have more than twenty icons on your desktop, I am betting that you are
only clicking on four or five of them regularly.

For all these reasons, I like to keep my desktop relatively uncluttered. 
All the programs are neatly gathered in the Quick Launch bar and the few icons
that you can see on my background are usually temporary files that I need to
refer to either a lot, or just for the current week.

With that in mind, I recently set out to modify my Mac Book Pro desktop in
order to show my favorite
background:  solid black.  And I mean black as in 0x000000,
which is the only background color that allows me to identify icons and
texts without any impediments.

So I right-clicked on the background, selected "Change Desktop Background…" and I was presented with a dialog box offering me
various options, among which "Solid colors", which I promptly selected. 
The dialog shown above appears, and here is my first surprise:  it looks
like I have a choice of ten preset backgrounds, but solid black is not one of
them.  "No problems", I think, I’ll just find the color chooser and I’ll
set the color myself. 

Problem is…  there is no such thing.

That’s right:  no color chooser.

Mac OS X won’t let you set your background to an arbitrary solid color: 
you have to choose one that Apple decided for you.  And by the way, did you
notice the dotted rectangle above?  It represents a solid white background. 
The trouble is…  this dotted line is not shown in reality (it was added
manually to this picture).  The background is there for you to select, but
it’s completely invisible because shown as white on white.  How silly is
that?

Anyway, back to my problem.

I can’t select my own background color.  That’s pretty retarded. 
You know, I can almost imagine what went on in the meeting room when this
decision was made:

Steve Jobs:  "Okay, I want the following ten choices for
background colors:  dark gray, solid aqua blue, …."
UI designer:  "Great, we’ll give the user a quick way to select
the colors that work best with our Aqua theme.  Where shall we put the
color chooser?"
"There shall be no color chooser."
"Sir?"
"There shall be no color chooser."
"But…  What if they don’t like any of these preset colors?"
"I don’t care.  The Mac OS desktop is the quintessence of beauty. 
I don’t want any of our tasteless users to ruin it with their own crappy
colors."
<uncomfortable silence, then forced grin> "Ah…  good one sir… 
Very funny.  Now, seriously…?"
"I was being serious."

So you can’t choose your own background color on Mac OS.  That’s the way
it is.  Of course, there are workarounds, such as creating a 1×1 image and
use it as a tiled background as explained
here,
but if I hear one more time that Mac OS is the most advanced desktop currently,
I’ll simply challenge the zealot to set the background color of their desktop to
solid magenta and watch her suffer in agony while I enjoy my black Windows
desktop which took me all of three seconds to configure.

 

Dynamic language, refactoring IDE. Pick one.

In a recent entry, Tim Bray revisits the old debate about refactoring dynamically typed languages:

IDEs all have a “Find Usages” feature, which in itself is handy (I’m thinking of nuking this stupid method, where is it used?) but if you think closely, is also vital too pretty well every other refactoring operation. If you’re going to rename a method or a class or extract an interface, it’s absolutely vital that the IDE know all the places where that code is used; otherwise refactoring equals breakage.

Of course, the problem is harder with dynamic languages, because variables aren’t declared, and because you can screw around with the insides of classes at runtime,

As always in this kind of debate, a commenter was quick to point out that “Smalltalk already did it”:

As with everything else: it’s all be done before: Smalltalk. In fact refactoring was pioneered (like many other things, oop, gui, unit testing, agile/xp …) in Smalltalk!
Smalltalk has 30 years of experience making the Ruby object model fast.

First of all, that commenter is wrong. All the Smalltalk IDE did for renaming was search and replace, which is, frankly, the best it could do. And clearly unacceptable today.

It’s simple, really: dynamic languages that are not statically typed (i.e. let you get away with not typing variables) simply *cannot* do certain refactorings, among which “renaming”.

How could they? They just don’t have the type information to know exactly which types are being used to invoke your methods.
A few months ago, I offered the following code snippet to the author of the Ruby Refactoring Browser:

def f1(o)
    o.init
end
def f2(o)
    o.init
end
class C
    def init
        ...
    end
end

And I asked him: “If I rename C.init to C.init2, how do you know which o.init must be renamed in f1 and f2?”.

His response was unequivocal:

This problem is difficult for dynamically typed language. I think computer can’t determine whether those must be renamed or not.

Therefore Ruby Refactoring Browser provides two functions, one is renaming all methods that have same name, and another is renaming only methods and calls that cleary belong the class. The former renames o.init in f1 and f2, and the latter doesn’t rename them.

This shouldn’t come as a surprise. One of the main reasons why people like dynamically typed languages is precisely because they don’t have to specify these types. Is it really so shocking that this convenience comes at the price of not being able to perform any operation that requires type information on this code?

As for Tim’s solution to this problem, which consists in having the IDE gain this knowledge by running your code instead of analyzing it statically: it’s clearly not going to work either.

In order for this approach to work, you would need to have 100% coverage with your tests which, I claim, is probably achieved by 0.001% of the projects out there, all of which are simple “Hello world” programs. And without this, who wants an IDE that performs a correct refactoring “most of the time”?

So is there any way out of this conundrum? I think there is: dynamic languages that allow you to type your variables if you feel like it, and the only language that I can think of that does that at the moment is Groovy. This is clearly the best of both worlds, because you only need to type your variables once (i.e. in your method signature) and type inference will save you the trouble of duplicating this information from that point on.