Java 5 was released in 2004 and introduced generics to the language using a mechanism called “erasure”. In the following couple of years, a lot of discussions took place comparing this approach to its counterpart, usually referred to as “reified generics”. The discussions then tapered down for a few years, until recently when two languages brought this topic back on the scene by supporting reified generics. These two languages are Gosu (created and used in production by GuideWire) and Kotlin (under development, created by JetBrains).

There is a lot of literature available on the subject of erasure and reified generics, but I thought I would take a few minutes to summarize the current state of the world and share a few thoughts on the pros and cons of each approach.

Let’s start with a few concrete examples. Here are snippets of code which do not compile under an erasure system but which would work fine with reified generics. If you are not familiar with the issues involved, here is a short rule of thumb that should allow you to understand what is going on: whenever you see a generic type, replaced it with Object (since that’s exactly what’s happening behind the scenes):

Overloading

public class Test<K, V> {
  public void f(K k) {
  }
  public void f(V v) {
  }
}

T.java:2: name clash: f(K) and f(V) have the same erasure
  public void f(K k) {
              ^
T.java:5: name clash: f(V) and f(K) have the same erasure
  public void f(V v) {

The workaround here is simple: rename your methods.

Introspection

public class Test {
  public <T> void f() {
    Object t;
    if (t instanceof List<T>) { ... }
  }
}

Test.java:6: illegal generic type for instanceof
    if (t instanceof List<T>) {}

There is no easy workaround for this limitation, you will probably want to be more specific about the generic type (e.g. adding an upper bound) or ask yourself if you really need to know the generic type T or if the knowledge that t is an object of type List is sufficient.

Instantiation

public class Test {
  public <T> void f() {
    T t = new T();
  }
}

Test.java:3: unexpected type
found   : type parameter T
required: class
    T t = new T();

This case is also a bit tricky but since it is, in my experience, more common than the others, I’ll spend a little more time discussing it.

As mentioned above, the virtual machine has no knowledge about the type T, which it only sees as an Object, so it won’t allow you to create an instance of it.

This is a good thing.

From a type standpoint, you know nothing about the T so you shouldn’t be able to instantiate it with the amount of knowledge you have. For all you know, T could be an abstract class, an interface or a class with no default constructor. In this case, type erasure keeps you honest by limiting what you can do on T to the operations that it knows about.

If you need to manipulate T, you will have to enable this through the type system. Do you need a new instance? Pass an additional Factory<T>. Do you need to call query()? on it? Create a type that contains this method and constrain T with it.

In 2006, Neal Gafter came up with a clever idea to make it possible to instantiate such erased types. He dubbed the technique “super type tokens”, and like most good ideas, it’s extremely simple: you force the creation of an anonymous class that contains the generic type, which you can then retrieve by a clever use of the introspection API.

Here is an example:

abstract class TypeReference<T> {}
public class TT {
  public static <T> void f(TypeReference<T> t) {
    ParameterizedType pt = (ParameterizedType)
        t.getClass().getGenericSuperclass();
    System.out.println(pt.getActualTypeArguments()[0]);
  }
  public static void main(String[] args) {
    TT.f(new TypeReference<String>() {});
  }
}

This will print "class java.lang.String" on the console, even though the method printing it is completely generic. The trick is on the highighted line: notice the empty braces, which create an anonymous instance of TypeReference, setting the stage for the introspection code in the method to retrieve the generic type.

Shortly thereafter, Bob Lee picked up this feature, fleshed it out and included it in Guice under the name TypeLiteral. Scala supports a similar feature called Manifest.

Ever since Java 5 came out, and despite my initial fears, I can’t say that I have been bothered much by the absence of type information in Java generics, and I am tempted to generalize this observation to the general Java population. Erasure turns out to have quite a few advantages and as it turns out, reified generics come with their own set of issues. Here are some of them.

The main problem is that reified generics would be incompatible with the current collections. In binary form, for sure, and probably in source form as well (we would want to distinguish between collections making used of reified types from their older counterpart). Rewriting would probably be mostly a matter of copy/pasting, except for the parts that make use of introspection and which would need to be adjusted. The generated byte code would also contain more information. For example, the following test:

o instanceof List<String>

would now test that the object is an instance of List but also that its elements are of type String. That’s quite a bit more work.

The extra type information also impacts the interoperability between languages within the JVM but also outside of it. For example, Scala recently announced some progress on its .Net compiler, which contains the following caveat:

The key limitation for the moment is that Scala programs cannot use libraries in .Net that are compiled using CLR generics, such as the .Net collections.

This is just a consequence of the fact C# has reified generics, and bytecode containing this supplemental type information requires more work to be parsed and converted in a form suitable for the client, as opposed to a simple List type.

So, where does this leave us?

Erasure has proven to work quite well for Java, and actually for quite a few other languages as well. Besides the two languages that I named above, there are only two other popular languages that support reified generics: C# and C++. All the others use erasure of some sort, and overall, it’s hard to argue that either approach brings a significant improvement in ease of use.

All in all, I am pretty happy with erasure and I’m hoping that the future versions of Java will choose to prioritize different features that are more urgently needed, such as closures or an improved module system.

Oh, and happy Java 7 day everyone!

As you probably saw by now, JetBrains just announced that they are working on a brand new statically typed JVM language called Kotlin. I am planning to write a post to evaluate how Kotlin compares to the other existing languages, but first, I’d like to take a slightly different angle and try to answer a question I have already seen asked several times: what’s the point?

We already have quite a few JVM languages, do we need any more?

Here are a few reasons that come to mind.

1) Coolness

New languages are exciting! They really are. I’m always looking forward to learning new languages. The more foreign they are, the more curious I am, but the languages I really look forward to discovering are the ones that are close to what I already know but not identical, just to find out what they did differently that I didn’t think of.

Allow me to make a small digression in order to clarify my point.

Some time ago, I started learning Japanese and it turned out to be the hardest and, more importantly, the most foreign natural language I ever studied. Everything is different from what I’m used to in Japanese. It’s not just that the grammar, the syntax and the alphabets are odd, it’s that they came up with things that I didn’t even think would make any sense. For example, in English (and many other languages), numbers are pretty straightforward and unique: one bag, two cars, three tickets, etc… Now, did it ever occur to you that a language could allow several words to mean “one”, and “two”, and “three”, etc…? And that these words are actually not arbitrary, their usage follows some very specific rules.

What could these rules be? Well, in Japanese, what governs the word that you pick is… the shape of the object that you are counting. That’s right, you will use a different way to count if the object is long, flat, a liquid or a building. Mind-boggling, isn’t it?

Here is another quick example: in Russian, each verb exists in two different forms, which I’ll call A and B to simplify. Russian doesn’t have a future tense, so when you want to speak at the present tense, you’ll conjugate verb A in the present, and when you want the future, you will use the B form… in the present tense. It’s not just that you need to learn two verbs per verb, you also need to know which one is which if you want to get your tenses right. Oh and these forms also have different meanings when you conjugate them in past tenses.

End of digression.

The reason why I am mentioning this is because this kind of construct bends your mind, and this goes for natural languages as much as programming languages. It’s tremendously exciting to read new syntaxes this way. For that reason alone, the arrival of new languages should be applauded and welcome.

Kotlin comes with a few interesting syntactic innovations of its own, which I’ll try to cover in a separate post, but for now, I’d like to come back to my original point, which was to give you reasons why you should be excited about Kotlin, so let’s keep going down the list.

2) IDE support

None of the existing JVM languages (Groovy, Scala, Fantom, Gosu, Ceylon) have really focused much on the tooling aspect. IDE plug-ins exist for each of them, all with varying quality, but they are all an afterthought, and they suffer from this oversight. The plug-ins are very slow to mature, they have to keep up with the internals of a compiler that’s always evolving and which, very often, doesn’t have much regards for the tools built on top of it. It’s a painful and frustrating process for tool creators and tool users alike.

With Kotlin, we have good reasons to think that the IDE support will be top notch. JetBrains is basically announcing that they are building the compiler and the IDEA support in lockstep, which is a great way to guarantee that the language will work tremendously well inside IDEA, but also that other tools should integrate nicely with it as well (I’m rooting for a speedy Eclipse plug-in, obviously).

3) Reified generics

This is a pretty big deal. Not so much for the functionality (I’ll get back to this in the next paragraph) but because this is probably the very first time that we see a JVM language with true support for reified generics. This innovation needs to be saluted.

Correction from the comments: Gosu has reified generics

Having said that, I don’t feel extremely excited by this feature because overall, I think that reified generics come at too high a price. I’ll try to dedicate a full blog post to this topic alone, because it deserves a more thorough treatment.

4) Commercial support

JetBrains has a very clear financial interest in seeing Kotlin succeed. It’s not just that they are a commercial entity that can put money behind the development of the language, it’s also that the success of the language would most likely mean that they will sell more IDEA licenses, and this can also turn into an additional revenue stream derived from whatever other tools they might come up with that would be part of the Kotlin ecosystem.

This kind of commercial support for a language was completely unheard of in the JVM world for fifteen years, and suddenly, we have two instances of it (Typesafe and now JetBrains). This is a good sign for the JVM community.

5) Still no Java successors

Finally, the simple truth is that we still haven’t found any credible Java successor.

Java still reigns supreme and is showing no sign of giving away any mindshare. Pulling numbers out of thin air, I would say that out of all the code currently running on the JVM today, maybe 94% of it is in Java, 3% is in Groovy and 1% is in Scala. This 94% figure needs to go down, but so far, no language has stepped up to whittle it down significantly.

What will it take? Obviously, nothing that the current candidates are offering (closures, modularity, functional features, more concise syntax, etc…) has been enough to move that needle. Something is still missing.

Could the missing piece be “stellar IDE support” or “reified generics”? We don’t know yet because no contender offers any of these features, but Kotlin will, so we will soon know. Either way, I am predicting that we will keep seeing new JVM languages pop up at a regular pace until one finally claims the prize. And this should be cause for rejoicing for everyone interested in the JVM ecosystem.

So let’s cheer for Kotlin and wish JetBrains the best of luck with their endeavor. I can’t wait to see what will come out of this.

Oh, and secretly, I am rooting for Eclipse to start working on their own JVM language too, obviously.

Link to the discussion on Google+