In
this TheServerSide thread, Amin Mansuri has this to say about debuggers:

I dunno.. maybe I’m a bit old school, but I’m influenced by Steve McConnell
author of Writing Solid Code and Debugging the Software Development process who
suggests that you should step through all your code in the debugger before
checking it in.

Yes!

I actually got this habit from reading "Writing Solid Code" myself. 
Single-stepping through all your new code might be a bit
overkill, though, but I am a firm believer in the idea that debuggers should be
used on new code, and I find myself doing that very often as the result of
reading this book more than ten years ago.

Frankly, I don’t understand the argument "unit tests are good enough, I
don’t need a debugger" put forth by so many anti-debugger people.

First of all, even unit tests break and have bugs, so you have to debug at
some point.

But more importantly, unit tests only give you a superficial view of the
quality of your code.  Okay, the bar is green, but are you really sure your
code is behaving the way you designed it, and not by accident?  Or that two
bugs are not canceling each other out, yielding a good result by accident?

What better way is there to really make sure of this than by putting a breakpoint in
your fresh new code, single-stepping through it and verifying that all the
variables are what you expect?

The best way to test a testing framework is, of course, to use it on itself. 
There are plenty of benefits to this approach:

• I like to "drink my own champagne" (I prefer this analogy to "eating my
  own dog food", being French and all).  If I don’t use my own code
  in-house, I am missing an opportunity to receive important feedback.
   
• It’s a great way to refine the listening API and make sure it covers all
  possible cases (this is the API that
  allows clients to be informed of when a test is being called, is succeeding,
  failing, etc…).
   
• It also allows me to make sure that TestNG is reentrant, since it is
  basically calling itself.  This is important since TestNG supports
  parallel execution of tests, so no deadlocks are permitted.

With this in mind, the requirements for testing TestNG are as follows:

• I need to be able to specify testng.xml programatically.  I don’t want
  to create sub XML files for each test.  Instead, I want to set up my
  testing environment completely in Java.
   
• When I run the tests, I want to make sure that the right methods are
  called and that their status (pass, fail, skip) matches what I expect.

The design I have achieved so far allows me to write tests like this:

@Test
public void partialGroupsClass() {
addClass("test.sample.PartialGroupTest");
addIncludedGroup("classGroup");
run();
String[] passed = {
"testMethodGroup", "testClassGroup"
};
String[] failed = {
};
verifyTests("Passed", passed, getPassedTests());
verifyTests("Failed", failed, getFailedTests());
}

In this test, I specify the test class to be invoked and the group to be
included for this run.  Then I run the tests and compare the methods
actually run against the ones I expect to be run.

As you can expect, the class PartialGroupTest contains several methods, of
which only testMethodGroup() and testClassGroup() belong to the group "classGroup",
so I only expect these methods to be invoked and to pass.

Now, how does this work exactly?  It’s quite simple:  the
inner TestNG instance is configured in a "beforeTestMethod" method, so that
it gets invoked before each test method.  This way, I am guaranteed to
start with a clean inner TestNG instance for each test method:

@Configuration(beforeTestMethod = true)
public void methodSetUp() {
setPassedTests(new HashMap());
setFailedTests(new HashMap());
setSkippedTests(new HashMap());
// ...
}

This method is defined in a base class that all my tests inherit, so I don’t
need to worry about this when I write my tests.  It is also responsible for
setting itself as a listener to the inner TestNG instance so that it gets
notified each time a test passes, fails or skips.  Later, the method
verifyTests() can be invoked by each test method to compare the methods run
against the ones they expect.

This infrastructure makes it trivial for me to add new tests and new
functionalities and be test-happy.  For example, as I was writing this
entry, I realized I could also test failures of partial groups, so I added a
couple of methods that belong to the same group, made them fail with an "assert
false", verified that the test failed, then I added the names of these two
methods in the failed array and the test passed.

I cannot over-emphasize how important it is to have a testing framework that
is flexible and yet, easy to use.  No matter how you look at it, if you
don’t understand how your tests work and you are not clear on how to add tests
yourself, your work as a developer is sub-optimal and you are probably not as
productive and not as aggressive as you could be.

This applies to "real life" as well, where you are working with an entire QA
department that came up with its own testing framework and is responsible for
writing most of the tests.  If this testing framework is too complex for
you and your developers, you need to get together with your QA department and
come up with a way that both units can exchange tests easily.

Debuggers are back in the spotlight with a recent post from Kasia.  She likes debuggers, so do I.

I have already expressed my views here and here on the importance of debuggers and my philosophy is quite simple:

No matter how productive you are with your current tools, I guarantee you will be more productive with an IDE and with a debugger.

I am pretty good with emacs and vi, with pinpointing potential bug areas by just reading code or quickly reading reams of output on a console, but at the end of the day, I just get things done faster if I add my IDE and a debugger to my arsenal of tools.  Ignoring this simple common sense is like being a caveman saying that he’s okay without fire.

The point of this entry is not to revive this debate (let’s leave cavemen into their caves) but to ponder the following question:  why is there almost no information on how to use a debugger wisely?  I don’t mean the user manual telling you how to set a breakpoint and how to single-step through code.  I mean a "good practice" guide.

As far as I know, this is not taught in school and hardly broached in books and I wonder why.  Is it because debuggers vary widely in scope and functionalities?

 I don’t believe this statement is true any more:  these days, debuggers have converged toward a very similar set of functionalities and except for a few innovations such as back-in-time debugging, you can reasonably expect your debugger to offer the same functions, regardless of the language or the operating system you are debugging in.

I would love to see a central place where all debugger tips and practices can be found, I’m sure there is a wealth of information waiting to be uncovered out there.

I’ll start with a few tips of my own.

Run method till it exits

Sometimes, you accidentally step into a method you didn’t mean to investigate, typically a library method.  You want to "get out" of this method so you can resume the debugging of the method you are really interested in.  You do this in Eclipse with the Step return button:

Press it and the current method will be run until it exits and the current frame is popped, taking you back to where the call was made.

Using conditional breakpoints.

I cannot overstate how important conditional breakpoints are.  Unless you are working on a very simple piece of code, the bug you are trying to find or the code you are trying to test will probably not be exercised right away, or it will be run several times without the parameters you are expecting.  The only way to get there faster is to type in entire expressions in your conditional breakpoint text box.

Using break conditions for println

As I stated in the link above, I’m not a big fan of "println debugging", which is antiquated and actually quite dangerous at times.  That being said, a bug sometimes happens after a certain number of iterations which cannot be easily captured by a conditional breakpoint (see below).  Or maybe, you just want to have some information about the state of the program displayed on stdout so that when you hit your breakpoint, the console contains some context information.

How could you do this without inserting println’s in your code?

Simple :  use a breakpoint expression that will do this println for you.

For example, the expression above could simple be replaced with "System.out.println(methodName)".  Since this expression will never return true, the program will never stop, but it will display the value of methodName throughout the entire execution.

Now, that’s what I call an efficient use of println and since the logic is contained in the debugger and not your code, no risk of shipping with it.

Of course, you don’t have to limit this to just println, you could just as easily modify existing variables or do some other crazy stuff…

How about you?  Any hot tips to share about your relationship with your debugger?

I recently added a couple of features to TestNG that make testing
asynchronous code (such as JMS) very easy:  parallelism and time-outs.

Parallelism instructs TestNG to run your test
methods in different threads. 
The trick here is that some of these test methods depend on each other, so these
methods
will obviously be invoked sequentially in the same thread.  But all the
other test methods that do not depend on anything else will be run in a separate
thread, picked from a pool (which size is configurable).  Here is an example
configuration:

<suite name="Main Suite" parallel="true" thread-count="10">
<test name="First Test">
<classes>
<class name="test.Test1" />
<class name="test.Test2" />
</classes>
</test>
</suite>

After the run, you can use the
chronological view to see which methods were
invoked and in what threads they ran.

Another interesting feature is time-outs, where you can indicate that a
specific test method is expected to return within a certain number of
milliseconds.  If it fails to do so, it will be interrupted and marked as a
FAIL with a TimeOutException.

With that in mind, testing JMS (or asynchronous code) becomes trivial. 
For example, imagine that our application posts a message on a JMS topic
whenever an error condition arises.  Here is how you could test it:

public class ErrorMessageTest implements MessageListener {
// Used to block
private Object m_done = new Object();
@Test(timeout = 5000 /* 5 seconds */)
public verifyErrorMessageGetsPosted() {
// register this class as a listener on the error topic
// create the error condition
// wait for completion
m_done.wait();
}
// implements MessageListener
public void onMessage(Message msg) {
if ( /* msg contains what we expect */) {
m_done.notify();
}
// else, do nothing, wait for another message
// or for the time-out to kick-in
}
}

This code is fairly self-explanatory and it leaves the dirty part (handling the
time-out and multi-threading issues to the testing framework).