Update: I posted a follow-up to this post

I recently finished reading “Programming Erlang”, by Joe Armstrong.

This book gives a pretty good overview of the language and the environment that surrounds it, and overall, I found it fairly enjoyable. I did get lost after a little while, though, but before I explain why, here is a quick review.

Diving in

My curiosity was piqued right from the start, where after explaining to you that the following statement assigns a value to a variable:

X=12.

This variable can then no longer be assigned a different value. In effect, all variables in Erlang are final. Now, this is intriguing and it certainly urged me to read further in order to understand

• How it was actually possible to write programs without variables.
• Why the language was designed this way.

I won’t spoil the surprise and the book does a very good job at demystifying this aspect of Erlang, but for those of you curious about the answer, I would summarize it as follows: Erlang uses an interesting mix of Unification (also called “pattern matching” and used in Prolog, although Erlang doesn’t have a “cut” operator) and of functional programming, as found in Lisp, Scheme and Haskell.

These names might be intimidating, especially when used in the same sentence, but I mean this as a compliment: Erlang manages to leverage these arcane concepts in ways that are actually fairly consistent and easy to follow. From that respect, Erlang might be a good language for beginners who want to learn more about unification and functional programming.

So far so good

The next chapter describes “sequential programming”, which is just a fancy name to delve into other aspects of Erlang that are still on fairly familiar grounds (at least from a functional programming standpoint). This chapter explains functions, list, list comprehensions (“fancy iterators”), lambda functions and records (Erlang’s version of structures). I wish this chapter had spent a little more time showing more examples of how these various functionalities interact with each other.

I found the next two chapters a bit ouf of place, both in terms of content (an entire chapter is dedicated to exceptions, which is actually an area where Erlang doesn’t differ that much from Java and other similar languages) but also on the examples. Chapter 5, called “Advanced Sequential Programming”, spends a lot of time showing how to manipulate bits and how to write a program that parses a binary image file format. I found these choices a bit odd, especially for a language that works at such a high level. I certainly wish the author had spent more time illustrating some of the other built-in functions shown in this chapter since it would most likely have helped the reader to get ready for what’s coming next…

Chapter 7, “Concurrency”, is the one I was eager to reach.

A new world of threads

If you’ve heard of Erlang these past months or years, it’s most likely because it has the reputation to make concurrent programming easier and faster than any other language so far. This is quite a provocative statement, and I was very curious to see how the language would live up to this claim.

What I find interesting in Erlang is that its functional aspect is a means to an end. The idea is that by using standard functional constructs, such as strict evaluation and single assignment, there is no mutable state in Erlang programs, which makes concurrency much easier to implement: imagine being able to spawn thousands of threads without ever having to resort to locking… This is exactly the premise of Erlang.

Avoiding mutable state is one thing, but you also need to have a way to communicate to other processes. This is typically achieved by sharing variables and carefully locking around them so you can’t thrash their value, but in Erlang, not only can processes not affect each other, they actually don’t have any visibility on their peers, not even their read-only state. When you want to transmit an information to a process, all you know is the id of this process and the data you want to send.

Erlang’s syntax to send messages is fairly simple:

Pid ! a_message

Pid is the process, a_message is any valid value (atom, variable, list, etc…). You receive a message with a structure that basically looks like a switch/case:

receive       % receive message sent to this process
% This is a tuple of a type atom and some data
{data, Data_content} -> do_something_else();
end.

Notice that Erlang’s pattern matching fits quite neatly with this paradigm, although I can imagine that such receive blocks can become quite huge as the program grows (I’ll come back to this problem later).

Erlang includes advanced functionalities to manage the most complex architectures made of hundreds of Erlang processes, among which: handling timeouts, spawning and finding processes, distributing them, notifying one or more processes when one of them dies, etc… Erlang also makes it trivial to send code and functions to remote processes, which opens up a lot of possibilities regarding hot-swapping or live upgrading.

The chapters that follow spend more time explaining how distributed programming works, how errors can be handled in massively distributed programs and tries to tackle the difficult problem of fault-tolerant code. They are followed by a few other utilitarian chapters covering the implementation of an irc client, linking to C code, the file API, sockets, ETS and DETS (Erlang databases, which are slightly different from SQL databases, as you can expect).

And then, we tackle the hardest part of this book: OTP and the generic server.

Confusion sets in

OTP stands for Open Telecom Platform, and it’s a set of libraries that were initially written by Ericsson (the author of the language) for their own telecom platform but which have now evolved to make it easy to build large-scale, fault-tolerant and distributed applications. This chapter starts by describing a “generic server” which will become the foundation of all the following examples.

It’s hard to explain exactly what gen_server does, but you can see it as a skeleton including the base logic to send and receive Erlang messages. This code serves actually a much greater purpose and the chapter eventually culminates by showing how a slight variant of gen_server makes it trivial to implement a server that can receive hot updates of the code that it’s running. This is quite impressive, to be honest, but I have to confess my skepticism on whether anyone would want to distribute hundreds of swappable servers on that many machines and then try to manage the upgrades in a way that makes it easy to trace and debug problems…

It’s approximately at this point that the book lost me. First of all, this chapter (and the ones that follow) contain some hardcore Erlang code, so it’s quite possible that it takes several reads (and probably more practical Erlang experience than I have) to understand the concepts better. And to be fair, the author warns us of that much at the very beginning of this chapter.

But what really got my head hurting was the vast amount of repetition that I found in this code. Every listing is basically a clone of the original gen_server code with just a few lines added, removed or altered. This made it very difficult for me to understand what was going on and, worse, to understand the philosophy behind these changes. On the contrary, I found myself having to parse the entire listing over and over again and detect the subtle changes that had been made. And as I progressed through the chapter, I had a clear sense that I was drifting further and further away from the big picture.

This, in turn, led me to wonder what was going on, and I started to realize that Erlang was lacking a fundamental concept that might explain why this code looked so hard to read: Erlang doesn’t have the concept of classes.

Scaling a runtime without scaling a code base

As far as I can tell from the book (please feel free to correct me if I’m wrong), Erlang has functions and modules, which contain a set of functions (each module can import other modules and needs to list all the functions it exports). But there is nothing in-between. I looked hard through the index of the book andon the web, but I wasn’t able to find anything that resembled to classes.

Let me clarify: the problem here is not the absence of classes but of what classes enable, among which polymorphism, inheritance and in more general terms, reusability. As I was going through these repeated listings, I found myself trying to find various ways to refactor them so that only the important part would be shown in the next listing, instead of the entire gen_server. But I couldn’t, and I don’t believe that it’s actually possible in Erlang.

This is a pretty big deal.

Fundamentally, these listings vary in their receive clause and in the various methods they implement. Refactoring a receive clause is like refactoring a switch/case: in general, not very convenient and even if you decide to replace it completely, you need to introduce polymorphism, which Erlang doesn’t seem to support at all. Similarly, it doesn’t seem possible to override an existing method and specialize it, nor is it possible to add a method to an existing module (or have a module extend an existing one). The closest you can come to with Erlang is by using multiple import statements, which is barely safer than #include.

In a nutshell: Erlang goes to great lengths to make it easy for you to distribute hundreds of pieces of code throughout an entire network, yet makes it nearly impossible to arrange this code in a meaningful way.

Looking for trouble

Another dark side of Erlang is error reporting. Here is what happens when you try to assign a variable two different values (probably the simplest mistake you could make):

4> X = 1234.
=ERROR REPORT==== 15-Sep-2007::20:12:01
Error in process <0.31.0> with exit value:
{{badmatch,1234}o,[{erlveeval,expr,3}]}
** exited: {{badmatch,1234},[{erl_eval,expr,3}]} **

This is pretty appalling. How can a beginner even remotely realize the mistake they just made? And trust me, it only gets worse from here.

Admittedly, once you start understanding how unification works, you painfully start to learn how to parse these error messages, because they usually all boil down to a failure to unify a variable with a pattern. Still, the whole exercise is very confusing and requires a lot more effort than it should. After eight years in existence, how come nobody has stepped up and introduced a friendly layer translating some of these error messages into something any human can understand, such as “Variable X already has value 1”?

Miscellaneous complaints

The main problem I have with Erlang at the moment is that despite the fact that it was created in 1998 (therefore, after Java, Ruby and Python), the language and its environment feel old. Very old. Most of the examples and listings are run from Erlang’s shell and it doesn’t seem that there is even a decent editor to help you. Besides the interesting functional/unification approach, the structure of the language is barely above that of C with hardly any means to organize or package your code in a meaningful way. For all I can see, an Erlang program is just a set of methods thrown in different modules, with barely any other interaction besides importing available to them.

The section about debugging in the book is merely a few pages long, and besides saying you can use “println debugging” or how to start the Erlang debugger (which seems pretty weak compared to what we’re used to), there is not much to be found there. Which I find very disappointing, because it seems to me that a language that encourages massive parallelism must be ripe for tough debugging problems. Unfortunately, the user will have to take their rite of passage themselves.

Another startling omssion of the book is testing. There is barely a word on how you can test an Erlang program, much less whether there exists a standard Erlang testing framework and, more interestingly, what kind of testing design patterns Erlang supports or requires.

Unfortunately for Erlang, functional programming and unification are a great fit for only a very tiny majority of problems. The fact that C, C++, Java, C# and Visual Basic are the dominant programming languages of our time is not a coincidence: their versatility and the imperative / OO philosophy have turned out to be extraordinarily adaptable and easy to learn and to extend.

No matter how much I admire Erlang’s ability to distribute processes across machines, the lesson we learned in the past decade in software engineering is that it’s easier to distribute data than code, and Erlang’s obsession at sending tiny pieces of beautiful and self-updatable code over the wire therefore appears to be completely misguided. It’s already hard enough to achieve this kind of distribution across a LAN, it’s simply impossible to achieve in any meaningful way on today’s Internet.

Conclusion

I enjoyed the book a lot, and I highly recommend it. It’s definitely at the same level of quality usually found on the Pragmatic Programmer’s bookshelf, and Joe Armstrong is doing a decent job at untangling and explaining a language that’s quite arcane for anyone not familiar with its underlying concepts. I’ll reserve my judgment on some of the sections where I got lost and I’ll do my best to try and reread them again in the near future, but I already know that the extensive repetitions in the listings make the endeavor difficult to tackle.

The main problem is Erlang itself, which is very interesting from an intellectual standpoint, but which I don’t see displacing any of the mainstream languages in a very long time…

So Apple pulled a fast one — again — and is taking the iPod to the next level with the iPod Touch. It’s undoubtedly a lovely gadget and a step in the right direction as it allows Apple to cleverly leverage all the technological advances they put in the iPhone.

However, the iPod Touch was not the highlight of Apple’s announcements. Predictably, there was quite a buzz when Steve Jobs said that the price of the iPhone was being slashed by $200. Potential buyers were delighted, Wall Street and early adopters… not so much.

While Apple has a documented track record of obsoleting its own technologies with regular announcements, the magnitude of the discount is a first. Whether people look at the $200 or 30% figure, everybody wonders if such a drastic reduction was necessary. After all, if the iPhone was selling so well, why not just discount it $100? And wait for the holiday season to slash it another $100?

Apple skeptics are quick to point out that this is an obvious sign that the iPhone is not doing so well and Apple fans retort that the iPhone is doing quite fine and is on target to meet Apple’s objectives. Fair enough, but we’re still talking about Apple’s “public” objectives. For all we know, they were still internally hoping for a bigger uptake and if they want to meet their 1% of the mobile market target for the end of 2008 (reminder: this represents approximately ten million iPhones sold by the end of 2008), they need to start getting very aggressive now. Even if 1% sounds like a modest goal, only a device under $150 has even a remote chance of hitting that goal, so I’m thinking we haven’t seen the last of brutal discounts.

On an lighter note, I find it amusing to see that some early buyers are quite angry to realize they could have saved $200 if they had waited a few months, and Apple zealots are quick to point out that this is business as usual coming from Cupertino.

Maybe early adopters will find some solace in the realization that this extra $200 bought them the right to brag about their new purchase on their blog, and maybe even make up for the loss by getting some extra money from their AdSense advertising…