Archive for September, 2007

The less tests, the better

Brian Doll posted a few interesting questions as a comment on my “Continuous tax” entry, which I’ll try to address in turn:

Oddly enough, some people see the act of writing tests as “continuous tax” in that it’s something else you have to write that does not go toward the “bottom line” of the features you’re implementing.

Actually, I don’t see writing tests as part of this continuous tax that I was describing earlier. While I agree that you usually write less tests in a statically typed language than a dynamically typed one, I think the difference in lines of code or number of tests is not significant enough to meaningfully impact the engineering cycle.

The continuous tax is triggered by the loss in explicitness that dynamic code usually suffers from. While it looks innocuous, the loss of typing actually has dire consequences on the maintainability and readability of your code by not giving you any hint on what type an object passed to a method really is and, worse, by making it impossible to apply even the simplest automatic refactorings such as renaming public functions (see this post for more details).

I’m interested to get your thoughts, as an author of a testing tool, on how you see the overall reliability of testing in dynamic and strongly typed languages.

As shocking as it may seem from the creator of a testing framework and the author of a book on testing, I actually always try to write as few tests as possible. The secret is to find the smallest subset of tests possible for a given functionality.

The way I see it, dynamically typed languages usually make me write the following tests:

  • Tests to verify type consistency.
  • Unit tests.
  • Functional tests.

Statically typed languages usually let me get away with writing:

  • Unit tests.
  • Functional tests.

To make matter worse, changing a type in my dynamic code forces me to update my tests manually, while the same operation in a statically typed language will not only be performed automatically by my IDE, it will also automatically update my business code *and* my test code (note that I haven’t mentioned TDD a single time because I believe that the practice of TDD is orthogonal to whether the language you are using is dynamically or statically typed).

In the past couple of days, a few people have chimed in with their private experiences of the continuous tax, among which Mark Derricutt and Eric Burke.

What’s interesting about a continuous tax is that you keep paying it all the time to the point that you might even forget about it. From that respect, I wouldn’t be surprised to see that developers who work with dynamic languages on a daily basis encounter the problems described by Eric and Mark regularly, but they just don’t notice any more: it’s just part of their routine. Java (and other statically typed languages) developers simply don’t ever pay this particular tax, so having to face these problems feels like a step backward in productivity (obviously, Java programmers pay different continuous taxes in other areas, but let’s save this topic for another day).

In a comment on Eric’s post, Weiqi wrote:

In that regard, the static typing works just like unit tests – it prevents you from breaking your code.

… which summarizes this issue pretty well: static typing gives you tests for free, and who would turn down free tests?

Continuous tax

Crazy Bob coined a term that I really like and that summarizes very neatly the pros and cons of using dynamically typed languages:

Brian, Java in general has a much higher learning curve than Ruby, especially when you consider generics and learning to get the most out of your tools. Java’s type system pays off in long term maintainability and usability. You can take a shortcut and skip this step with Ruby, and you’ll hit the ground running a little faster, but you’ll also pay a continuous tax.

This “continuous tax” is defined by the fact that when you need to maintain or use an API that was written in a language such as Ruby or Python, you have very little information available to you, and even if you eventually figure it out by looking at the sources of the tests (does anyone ever do that?), this knowledge you gain is ephemereal, and you will have to go through that same exercise if you need to modify this same portion of code a year later.

Another facet of the continuous tax is that by choosing a language that’s dynamically typed, you forego a set of tools (IDE’s) and practices (automatic refactorings) that have proven immensely beneficial to software maintenance and evolution. And beware anyone who tells you that not being able to use an IDE is a good thing, because you are most likely talking to a language zealot who will not carefully consider all alternatives before giving you their opinion.

Interestingly, Ruby and Ruby on Rails bigots seem to be quite oblivious to the art of maintaining software, probably because most of them are consultants and because software that’s hard to maintain means charging more consulting fees. Or maybe they just don’t need to see beyond a few months in the future of the code base they help develop.

At any rate, the term “continuous tax” to describe dynamically typed languages is dead-on accurate because it captures very precisely the trade-off you are making when you choose to use a language that is not statically typed.


Update: I posted a follow-up to this post

I recently finished reading “Programming Erlang”, by Joe Armstrong.

This book gives a pretty good overview of the language and the environment that surrounds it, and overall, I found it fairly enjoyable. I did get lost after a little while, though, but before I explain why, here is a quick review.

Diving in

My curiosity was piqued right from the start, where after explaining to you that the following statement assigns a value to a variable:


This variable can then no longer be assigned a different value. In effect, all variables in Erlang are final. Now, this is intriguing and it certainly urged me to read further in order to understand

  • How it was actually possible to write programs without variables.
  • Why the language was designed this way.

I won’t spoil the surprise and the book does a very good job at demystifying this aspect of Erlang, but for those of you curious about the answer, I would summarize it as follows: Erlang uses an interesting mix of Unification (also called “pattern matching” and used in Prolog, although Erlang doesn’t have a “cut” operator) and of functional programming, as found in Lisp, Scheme and Haskell.

These names might be intimidating, especially when used in the same sentence, but I mean this as a compliment: Erlang manages to leverage these arcane concepts in ways that are actually fairly consistent and easy to follow. From that respect, Erlang might be a good language for beginners who want to learn more about unification and functional programming.

So far so good

The next chapter describes “sequential programming”, which is just a fancy name to delve into other aspects of Erlang that are still on fairly familiar grounds (at least from a functional programming standpoint). This chapter explains functions, list, list comprehensions (“fancy iterators”), lambda functions and records (Erlang’s version of structures). I wish this chapter had spent a little more time showing more examples of how these various functionalities interact with each other.

I found the next two chapters a bit ouf of place, both in terms of content (an entire chapter is dedicated to exceptions, which is actually an area where Erlang doesn’t differ that much from Java and other similar languages) but also on the examples. Chapter 5, called “Advanced Sequential Programming”, spends a lot of time showing how to manipulate bits and how to write a program that parses a binary image file format. I found these choices a bit odd, especially for a language that works at such a high level. I certainly wish the author had spent more time illustrating some of the other built-in functions shown in this chapter since it would most likely have helped the reader to get ready for what’s coming next…

Chapter 7, “Concurrency”, is the one I was eager to reach.

A new world of threads

If you’ve heard of Erlang these past months or years, it’s most likely because it has the reputation to make concurrent programming easier and faster than any other language so far. This is quite a provocative statement, and I was very curious to see how the language would live up to this claim.

What I find interesting in Erlang is that its functional aspect is a means to an end. The idea is that by using standard functional constructs, such as strict evaluation and single assignment, there is no mutable state in Erlang programs, which makes concurrency much easier to implement: imagine being able to spawn thousands of threads without ever having to resort to locking… This is exactly the premise of Erlang.

Avoiding mutable state is one thing, but you also need to have a way to communicate to other processes. This is typically achieved by sharing variables and carefully locking around them so you can’t thrash their value, but in Erlang, not only can processes not affect each other, they actually don’t have any visibility on their peers, not even their read-only state. When you want to transmit an information to a process, all you know is the id of this process and the data you want to send.

Erlang’s syntax to send messages is fairly simple:

Pid ! a_message

Pid is the process, a_message is any valid value (atom, variable, list, etc…). You receive a message with a structure that basically looks like a switch/case:

receive       % receive message sent to this process
% This is a tuple of a type atom and some data
{data, Data_content} -> do_something_else();

Notice that Erlang’s pattern matching fits quite neatly with this paradigm, although I can imagine that such receive blocks can become quite huge as the program grows (I’ll come back to this problem later).

Erlang includes advanced functionalities to manage the most complex architectures made of hundreds of Erlang processes, among which: handling timeouts, spawning and finding processes, distributing them, notifying one or more processes when one of them dies, etc… Erlang also makes it trivial to send code and functions to remote processes, which opens up a lot of possibilities regarding hot-swapping or live upgrading.

The chapters that follow spend more time explaining how distributed programming works, how errors can be handled in massively distributed programs and tries to tackle the difficult problem of fault-tolerant code. They are followed by a few other utilitarian chapters covering the implementation of an irc client, linking to C code, the file API, sockets, ETS and DETS (Erlang databases, which are slightly different from SQL databases, as you can expect).

And then, we tackle the hardest part of this book: OTP and the generic server.

Confusion sets in

OTP stands for Open Telecom Platform, and it’s a set of libraries that were initially written by Ericsson (the author of the language) for their own telecom platform but which have now evolved to make it easy to build large-scale, fault-tolerant and distributed applications. This chapter starts by describing a “generic server” which will become the foundation of all the following examples.

It’s hard to explain exactly what gen_server does, but you can see it as a skeleton including the base logic to send and receive Erlang messages. This code serves actually a much greater purpose and the chapter eventually culminates by showing how a slight variant of gen_server makes it trivial to implement a server that can receive hot updates of the code that it’s running. This is quite impressive, to be honest, but I have to confess my skepticism on whether anyone would want to distribute hundreds of swappable servers on that many machines and then try to manage the upgrades in a way that makes it easy to trace and debug problems…

It’s approximately at this point that the book lost me. First of all, this chapter (and the ones that follow) contain some hardcore Erlang code, so it’s quite possible that it takes several reads (and probably more practical Erlang experience than I have) to understand the concepts better. And to be fair, the author warns us of that much at the very beginning of this chapter.

But what really got my head hurting was the vast amount of repetition that I found in this code. Every listing is basically a clone of the original gen_server code with just a few lines added, removed or altered. This made it very difficult for me to understand what was going on and, worse, to understand the philosophy behind these changes. On the contrary, I found myself having to parse the entire listing over and over again and detect the subtle changes that had been made. And as I progressed through the chapter, I had a clear sense that I was drifting further and further away from the big picture.

This, in turn, led me to wonder what was going on, and I started to realize that Erlang was lacking a fundamental concept that might explain why this code looked so hard to read: Erlang doesn’t have the concept of classes.

Scaling a runtime without scaling a code base

As far as I can tell from the book (please feel free to correct me if I’m wrong), Erlang has functions and modules, which contain a set of functions (each module can import other modules and needs to list all the functions it exports). But there is nothing in-between. I looked hard through the index of the book andon the web, but I wasn’t able to find anything that resembled to classes.

Let me clarify: the problem here is not the absence of classes but of what classes enable, among which polymorphism, inheritance and in more general terms, reusability. As I was going through these repeated listings, I found myself trying to find various ways to refactor them so that only the important part would be shown in the next listing, instead of the entire gen_server. But I couldn’t, and I don’t believe that it’s actually possible in Erlang.

This is a pretty big deal.

Fundamentally, these listings vary in their receive clause and in the various methods they implement. Refactoring a receive clause is like refactoring a switch/case: in general, not very convenient and even if you decide to replace it completely, you need to introduce polymorphism, which Erlang doesn’t seem to support at all. Similarly, it doesn’t seem possible to override an existing method and specialize it, nor is it possible to add a method to an existing module (or have a module extend an existing one). The closest you can come to with Erlang is by using multiple import statements, which is barely safer than #include.

In a nutshell: Erlang goes to great lengths to make it easy for you to distribute hundreds of pieces of code throughout an entire network, yet makes it nearly impossible to arrange this code in a meaningful way.

Looking for trouble

Another dark side of Erlang is error reporting. Here is what happens when you try to assign a variable two different values (probably the simplest mistake you could make):

4> X = 1234.
=ERROR REPORT==== 15-Sep-2007::20:12:01
Error in process <0.31.0> with exit value:
** exited: {{badmatch,1234},[{erl_eval,expr,3}]} **

This is pretty appalling. How can a beginner even remotely realize the mistake they just made? And trust me, it only gets worse from here.

Admittedly, once you start understanding how unification works, you painfully start to learn how to parse these error messages, because they usually all boil down to a failure to unify a variable with a pattern. Still, the whole exercise is very confusing and requires a lot more effort than it should. After eight years in existence, how come nobody has stepped up and introduced a friendly layer translating some of these error messages into something any human can understand, such as “Variable X already has value 1”?

Miscellaneous complaints

The main problem I have with Erlang at the moment is that despite the fact that it was created in 1998 (therefore, after Java, Ruby and Python), the language and its environment feel old. Very old. Most of the examples and listings are run from Erlang’s shell and it doesn’t seem that there is even a decent editor to help you. Besides the interesting functional/unification approach, the structure of the language is barely above that of C with hardly any means to organize or package your code in a meaningful way. For all I can see, an Erlang program is just a set of methods thrown in different modules, with barely any other interaction besides importing available to them.

The section about debugging in the book is merely a few pages long, and besides saying you can use “println debugging” or how to start the Erlang debugger (which seems pretty weak compared to what we’re used to), there is not much to be found there. Which I find very disappointing, because it seems to me that a language that encourages massive parallelism must be ripe for tough debugging problems. Unfortunately, the user will have to take their rite of passage themselves.

Another startling omssion of the book is testing. There is barely a word on how you can test an Erlang program, much less whether there exists a standard Erlang testing framework and, more interestingly, what kind of testing design patterns Erlang supports or requires.

Unfortunately for Erlang, functional programming and unification are a great fit for only a very tiny majority of problems. The fact that C, C++, Java, C# and Visual Basic are the dominant programming languages of our time is not a coincidence: their versatility and the imperative / OO philosophy have turned out to be extraordinarily adaptable and easy to learn and to extend.

No matter how much I admire Erlang’s ability to distribute processes across machines, the lesson we learned in the past decade in software engineering is that it’s easier to distribute data than code, and Erlang’s obsession at sending tiny pieces of beautiful and self-updatable code over the wire therefore appears to be completely misguided. It’s already hard enough to achieve this kind of distribution across a LAN, it’s simply impossible to achieve in any meaningful way on today’s Internet.


I enjoyed the book a lot, and I highly recommend it. It’s definitely at the same level of quality usually found on the Pragmatic Programmer’s bookshelf, and Joe Armstrong is doing a decent job at untangling and explaining a language that’s quite arcane for anyone not familiar with its underlying concepts. I’ll reserve my judgment on some of the sections where I got lost and I’ll do my best to try and reread them again in the near future, but I already know that the extensive repetitions in the listings make the endeavor difficult to tackle.

The main problem is Erlang itself, which is very interesting from an intellectual standpoint, but which I don’t see displacing any of the mainstream languages in a very long time…

Bragging rights

So Apple pulled a fast one — again — and is taking the iPod to the next level with the iPod Touch. It’s undoubtedly a lovely gadget and a step in the right direction as it allows Apple to cleverly leverage all the technological advances they put in the iPhone.

However, the iPod Touch was not the highlight of Apple’s announcements. Predictably, there was quite a buzz when Steve Jobs said that the price of the iPhone was being slashed by $200. Potential buyers were delighted, Wall Street and early adopters… not so much.

While Apple has a documented track record of obsoleting its own technologies with regular announcements, the magnitude of the discount is a first. Whether people look at the $200 or 30% figure, everybody wonders if such a drastic reduction was necessary. After all, if the iPhone was selling so well, why not just discount it $100? And wait for the holiday season to slash it another $100?

Apple skeptics are quick to point out that this is an obvious sign that the iPhone is not doing so well and Apple fans retort that the iPhone is doing quite fine and is on target to meet Apple’s objectives. Fair enough, but we’re still talking about Apple’s “public” objectives. For all we know, they were still internally hoping for a bigger uptake and if they want to meet their 1% of the mobile market target for the end of 2008 (reminder: this represents approximately ten million iPhones sold by the end of 2008), they need to start getting very aggressive now. Even if 1% sounds like a modest goal, only a device under $150 has even a remote chance of hitting that goal, so I’m thinking we haven’t seen the last of brutal discounts.

On an lighter note, I find it amusing to see that some early buyers are quite angry to realize they could have saved $200 if they had waited a few months, and Apple zealots are quick to point out that this is business as usual coming from Cupertino.

Maybe early adopters will find some solace in the realization that this extra $200 bought them the right to brag about their new purchase on their blog, and maybe even make up for the loss by getting some extra money from their AdSense advertising

“Next Generation Testing” book now available for preorder

Our book is now available for preorder on Amazon. The working title is “Next Generation Java Testing”.

It’s approximately five hundred pages of dense Java-related testing information illustrated by a lot of (mostly small) listings and code snippets. The Rough Cut version is also available on Safari.

It should be publicly available in November.