Archive for May, 2011

The home computer, now and then

This prediction of the future computer in 1966 is surprisingly accurate in many ways, if you can get past the sexist clichés.

Ironically, the only part of the video that we still haven’t realized today is the fact that I still need to fill my own taxes even though the IRS has all the information on its servers.

Imagine a world where the IRS fills your tax return for you, now that is some serious science fiction.

Forget BASIC, it’s about the magic

It is practically impossible to teach good programming to students that have had a prior exposure to BASIC: as potential programmers they are mentally mutilated beyond hope of regeneration.

This quote is from Edsger Dijkstra in a short article he wrote in 1975 called “How do we tell truths that might hurt?”.

Dijkstra had the reputation of being facetious and it’s hard to tell whether he really meant this or if he was just poking some harmless fun at the various technologies in vogue in 1975 (a practice I can certainly relate to). Whatever the case may be, his quote has been picked up and reproduced literally quite a bit, often by people trying to engage in some language snubiness warfare.

Personally, I think this quote is completely and unequivocably wrong.

Most of the developers that I know or that I have interacted with and who are in their thirties or above have, obviously, started with BASIC. A lot of them are very good engineers, and not just at one language in particular, but also very knowledgeable about the technology. They routinely write code in several languages and they are also very comfortable with peripheral notions such as databases, clustering, parallelism, etc… You name it. All of them would agree that while BASIC was quite primitive, it was also a great enabler. It allowed you to get things running on the screen very quickly and there is no doubt that it created an entire generation of people who became absolutely passionate about the field of software engineering.

BASIC should receive praises for this fact alone. But let’s not stop there.

Was BASIC really that primitive? Looking back in the days, I would say “far from it”. Granted, at the time, some languages were beginning to push structured and OO programming into the industry, but for a mainstream language that was available on every single computer that was sold at the time, I am tempted to say… “Loops, variables, rudimentary math and graphical functions… not bad!”.

I would also argue that every argument that you can make against BASIC can be made against assembly language, and in worse ways.

Is there anyone who thinks that assembly language corrupts the mind of programmers beyond redemption? Certainly not me. Most developers who learn assembly language will quickly move on to higher level languages while retaining a very accurate mental model of what’s going on under thehood, which is a win-win situation in my opinion. Just because you know the difference between stack and heap and you still remember a few hexadecimal opcodes from your favorite microprocessor doesn’t mean you won’t be able to understand classes, polymorphism or closures.

I certainly loved my time writing BASIC, and while it ignited my passion for this industry, the epiphany for me came in a slightly different form.

Story time.

In the first year with my ever beloved Apple ][, I was writing a lot of Applesoft BASIC but probably reading even more of it. Back then, there was no Internet and magazines were few and far between, so I took any opportunity I could to read code.

At some point, I came across a mysterious instruction called “CALL 768”. I had read the documentation for this CALL instruction but I still didn’t understand what it did. Puzzled, I typed “CALL 768” at the prompt and something magical happened: the computer played a short tune.

Obviously, I knew the Apple ][ could play music but I was startled to discover that you could do so with such a simple command. I quickly added it to my own program and I was delighted to find out that it worked. A few days later, I resumed working on my program but as soon as I launched it, it froze. I quickly identified “CALL 768” as the source of the problem. I started suspecting that the previous program I was investigating had put something in memory, which was wiped when I turned the computer off. To verify this hypothesis, I launched this other program, then launched mine again and this time, it worked.

Mystery solved.

Well, partially. My curiosity was piqued and now, I just had to figure out what was going on. I started wondering where in the memory this code could be. It obviously wasn’t in BASIC since a LIST didn’t show anything. My (still quite nascent) hexadecimal knowledge kicked in and I realized that 768 was $300 in hexadecimal. I had some rudimentary knowledge of something called the “Monitor”, which allowed you to read machine language. I was, however, unable to really understand it, a bit like knowing the Hiragana alphabet, being able to pronouce the words that Hiragana symbols form but being unable to understand them.

The instruction to start the monitor was “CALL -151” (mmmh… there’s this CALL instruction again). Once there, you can see the content of the memory by typing the address followed by “L”. Feverishly, I typed “300L” and sure enough, an assembly listing was there. I rebooted, checked the location again, the assembly was gone. I ran the other program and the assembly returned. I didn’t know how the assembly went there (I would investigate this later) but I knew I was well on my way to figuring this puzzle out.

It took me several days of painful trials and errors, but this little investigation led me to take my first steps in 6502 programming and more importantly, to understand how BASIC and assembly language could interact with each other. The realization that BASIC could read and write memory with PEEK and POKE was the last missing piece, and this understanding blew whatever was left of my mind at the time. I was just amazed at how beautiful this orchestration was, and how all the pieces fitted so neatly together.

That was it. This is what started the passion in me. It wasn’t a language, it wasn’t a specific hardware or a concept. It was the realization that despite all the apparent magic, everything was logical and no mystery could resist meticulous investigations and logical thinking. I still felt like a complete beginner but at that moment, I knew that given enough time and effort, I should be able to puzzle it all out. And maybe be able to create all these wonderful programs that I had been using so far without understanding much of how they worked.

BASIC had very little to do with it. It was never about BASIC, it was always about the magic.

Amazon Cloud Music

I’ve been using Amazon’s Music Cloud for a couple of weeks and so far, I’m pretty happy. It works largely as advertised but is clearly a very early version which could use a lot of improvements.

I picked the 50 gig plan for $50. The process was easy and it took about fourteen hours to upload my entire collection (since it’s mostly a one time operation, I really don’t understand why anyone would have a problem with this) Amazon allows me to re-download my own music one album at a time but not download entire artists, much less the entire collection. I can’t say I’m very surprised since it’s always been notoriously difficult to get Amazon to allow me to re-download the MP3’s I buy from them (I basically had to file tickets, and even then, it was a struggle). Amazon obviously doesn’t want to be seen as a music back up service.

The music player itself is probably as minimalistic as you can get. You can display your music by songs, albums, artists and genres. Playback can shuffle and repeat and Amazon did a decent job at finding the album covers for most of my collection.

Here is a list of the (minor) annoyances I have found so far:

  • Periodic sign off. For some irritating reason, Amazon forces me to sign in again once a day or so (I didn’t time it precisely). It’s not the end of the world but annoying.
  • No keyboard shortcuts. There were several times where I needed to do some repeated operations on my collection (such as deleting duplicates) and having to use the mouse for it was painful. I would love to get Gmail shortcuts to manipulate my collection.
  • Bad single song visualizer. The box showing the current song and album is poorly laid out, with small fonts and chopped up words, and since there is no way to change its lay out or alter column widths (same problem with the collection view), it’s as bad as Windows Media Player.

Be aware that Amazon uses MP3 tags exclusively to recognize your collection. I know it sounds obvious, but it took me a while to understand why moving folders around on my local hard drive and then re-uploading the songs produced no changes. The upside is that I finally got a good incentive to do some MP3 tag clean up in my collection. Once I fixed this, my music library on Amazon looked much better.

I would really have liked this initial version to contain a de-duplicator. Looking at my collection once it was uploaded made it clear that a lot of the songs were there in duplicates (and sometimes more, especially if you do several uploads of the same album with slightly different tags).

Overall, it’s pretty clear to me that what Amazon really needs is an API for their Music Cloud. And fast.

We will soon have three big players in this market (Amazon and Google today, Apple soon) and the distinction will certainly not be on what the web page looks like or what the player can do. Instead, users will most likely look at the surrounding ecosystem, which includes both what the company offers (Amazon and Apple sell music, so they have an advantage over Google in terms of easy integration) and also what third party developers will be able to build on top of the platform.

A lot of of gripes I listed above would most likely be solved in a matter of days by the community if a Music API were available. This would also open the door to having native clients which would completely remove the dependency on having a web browser open to play your music. Obviously, I can imagine that the RIAA will do its best to kill or at least slow down the advent of such API’s, but since Amazon and Google have already shown they were not afraid to proceed without their agreement, I’m still hopeful it will happen soon.

Open type systems

java.lang.Class is probably one of Java’s most famous classes, and whether you write applications or frameworks, you have probably encountered it many times, either superficially (getClass()) or in more depth (reflection, class loaders, etc…).

java.lang.Class is also a very special class in the Java ecosystem, and as such, it is limited in some very specific ways. One of them is that developers can’t create a new Class object themselves (the class has no public constructor), only the JVM can create Class objects. When you are writing Java code and you want to materialize some of these objects, your options are fairly limited:

  • Create a class loader and use the defineClass() method. Class loaders are complex and require a lot of intimate knowledge of the JVM to get right, and defineClass() also requires you to supply the bytecode for the class you want to create.
  • Code generation. This is the more traditional approach. It’s easier to achieve in some ways (all you need to do is generate the source of the class you need) and more complicated in others (you need to infect your build with the knowledge of these generated classes and teach it how to bundle them with your application).
  • You can use dynamic proxies. Dynamic proxies don’t exactly materialize new classes, so they shouldn’t really be part of this list, but I thought I’d mention them since they can be a pretty powerful solution to the overall problem of generating new types in Java programs (note that this approach is less typed than the other two).

Despite their shortcomings, all these approaches have been used with great success by various Java frameworks, and these techniques played a great role in making Java such a strong platform that remains able to solve most of the engineering problems that arise these days.

Having said that, what can we do to improve on these approaches?

Imagine a new type called IType. This is actually an interface, so any class can implement it. Let’s now assume that everywhere in the JVM where a Class is expected, we can use an IType instead (obviously, we’ll make Class implement IType). IType is the new representation of a class inside the JVM, and any object that implements it can be passed instead of a Class.

Obviously, an IType needs to be able to perform all the operations that Class can, among which:

  • Return information about its name, package, parent type(s), nested types, etc…
  • Respond to standard introspection operations, such as returning the list of methods that it exposes.

The methods returned by the IType need to be abstracted as well, so instead of returning java.lang.reflect.Method, we return a new type called IMethod. We place the same requirements on IMethod that exist on Method: introspection must work on these objects and, more importantly, we need to implement invoke(). If the object we are returning is an instance of Method, implementing invoke() is trivial (just delegate it to the underlying Method object).

But what happens if, instead of returning standard Java Class and Method objects, we decide to return something different?

Actually, what could these other objects be?

As it turns out, anything. They could represent files: .xml obviously, but how about .xsd or .properties? Or how about sources of another language? Maybe a .groovy, .bash or .scala? Thinking further outside the box, how about concepts that are not files, such as database tables? Or virtual files, or files present on a remote file system? How about the endpoint of a network connection (and we don’t care what’s feeding this endpoint)? We could also decide to represent the results of a REST or AJAX or JSON API.

This new type system doesn’t care what the underlying concept actually is, all it needs is a set of ITypes (and a few other details that I’m glossing over), and it will expose these types to the JVM as if they were genuine classes. Whether they are backed by Java bytecode or some other logic is irrelevant.

Here is a quick example of what such a type might look like.

Let’s suppose we want to make Java property files first class citizens in this new type system (it’s a silly example since a simple hash table is good enough, but this is just for illustration). The idea is that whenever we encounter a file called “foo.properties”, we’ll create a class called FooProperties and each key/value found in this file will be represented as a getter. For example, loading this file in this new type system:

# File: host.properties
host: 127.0.1.0
port: 1245

will allow you to write the following code:

HostProperties fp = PropertyTypeLoader.load("host.properties");
System.out.println("Host:" + fp.getHost() + " port:" + fp.getPort());

The implementation of the corresponding IType and IMethod interfaces is pretty straightforward in this case, but this should give you an idea of the flexibility of such a type system.

What I have just described is basically Gosu’s Open Type System. Gosu is not just “yet another JVM language”, it also enables the kind of type openness that allows you to materialize pretty much anything as a real Gosu type. Once you have described your type, it becomes a part of your Gosu program and as such, benefits from the same privileges as any other type (e.g. it can be used by IDE’s for completion, browsing, refactoring, etc…).

Having said that, let’s now put things in perspective:

  • Java’s metaclass design is intentional: it was conceived this way for security reasons, the idea being that types materializing in your JVM process need to go through a very specific pipeline so it can be validated and checked against malignant attacks. I am guessing that this system is extremely paranoid because Java was initially driven by applets, but I wonder if such concerns are still valid today.
  • While elegant, Gosu’s open type system is basically just saving you from a code generation phase.

I’m guessing that not everyone will think that the benefits from such a type system outweigh the drawbacks, but I’d love to hear some feedback on whether you think that future languages should give access to such functionality or if it’s just a nice theoretical toy whose practicaly is doubtful.

Here is some additional material on Gosu’s Open type System:

Google I/O 2011

Wow… what a keynote.

There is usually no disputing that Steve Jobs’ keynotes are the best in terms of delivery and content, but I have to say that Google IO’s keynote this year is the first time where I think that Google actually topped that. Jobs still reigns supreme in charisma and stage presence, but when it comes down to content, Google’s keynote this morning was, simply put, overwhelming.

I’m still digesting it all so I’ll just focus on the highlights right now.

The big announcement was, of course, Google Music. This is actually not the most impressive technological achievement of the morning, since the service seems to be extremely similar to Amazon’s and persistent rumors these past few days took away some of the surprise. However, I believe that Apple is now lagging behind in music cloud area for two reasons:

  • They still haven’t announced anything.
  • They still require a cable and iTunes to synchronize.

This last point is important. Not so much the requirement for iTunes, which doesn’t get a lot of love, even from Apple enthusiasts, but the fact that a cable is still required to transfer any kind of content to your iPhone makes Apple look like they are still living in the 90’s. It’s all the more surprising since Apple is claiming to usher us into the “post-PC” era, but they are now the only company still requiring a PC to manage your phone.

Apple is going to have to embrace over-the-air updates if they want to compete.

Back to the Google Music announcement, I’m looking forward to trying it but the only reason why I might be tempted to switch to it instead of using Amazon’s (like I’m currently doing) is if they allow me complete download access to my library once I have uploaded it on Google’s servers (Amazon doesn’t, as far as I know). Another advantage of Amazon over Google here is that Amazon lets you purchase music, and these purchases are automatically added to your cloud drive.

Also, it looks like Google wasn’t able to reach a licensing deal with the music majors but decided to launch their service anyway, following in Amazon’s footsteps. There are persistent rumors claiming that the reason why Apple still hasn’t launched their cloud service is because they are still trying to negotiate such deals. Time will tell whether this matters or not.

The other interesting point of the keynote was to see Andoid grow way, way beyond phones. I’ve always heard rumors of seeing Android on mundane devices such as photo frames or cars, but what I saw this morning makes me think that in a few years from now, I’ll be surrounded with Android devices in places that I would never have suspected. Work out stations, home appliances and the Arduino toolkit (very excited about this one) are just the beginning of the Android tsunami. I am definitely going to keep a very close eye on Project Tungsten, and if I were Sonos (which is an awesome product), I would be very nervous right now because Android is looking like the first very credible threat they have ever faced.

The perks of ubiquity

While watching TV last night, I saw Sprint’s Kyocera commercial (“the phone with two screens”) and I noticed something interesting: this phone runs Android but neither the name “Android” nor the cute green robot icon appear anywhere. Not even in small prints at the end.

This might not be the first commercial of a mobile phone omitting to name the operating system it runs on, but I think this is pretty significant for the Android brand: it means that Android has moved from the status “dominant” to the more coveted “pervasive” category.

When was the last time that you watched a commercial from HP, Dell or Lenovo that mentioned the operating system it’s running on? Right, never. Okay, maybe they do mention Windows because they have to specify which version of Windows you will be getting (e.g. Vista or Windows 7) but there is no doubt in the mind of the viewers of these commercials that the operating system is Windows.

Pervasiveness comes with a lot of perks whose value is very often undervalued.

For starters, free commercials. Every time a device running Android is being advertised on TV, radio or on the Internet, Google receives a free plug.

Repeated buzz is another benefit of pervasiveness. Apple releases one new phone a year (although there are rumors claiming this might soon become “two”). When they do, they undoubtedly receive a very high amount of coverage and hype, along with pictures of customers lining up around the block to buy the new device and providing enthusiastic testimonies about the greatness of Apple. But once the frenzy subsides, the iPhone brand soon recedes into obscurity and doesn’t receive much exposure until the next phone comes out, one year later. In contrast, Android is receiving free lip service and buzz on a regular basis, which contributes greatly to making sure that potential phone buyers are never further than a thought away from the word “Android”.

In contrast to the iPhone, you hardly see a lot of excitement for upcoming Android devices, but since new devices come out pretty much on a weekly basis, not a week goes by without the tech industry reporting on a new Android phone or tablet, showing close ups and screen shots, testing its camera, video and other features, comparing it to the iPhone, etc…

This buzz is limited in scope, though, because of its high frequency: you never really watch a commercial for an Android device that shows a revolutionary new feature (the Kyocera and its intriguing two screens is an exception to this observation) but this doesn’t prevent from slowly adding up, layer after layer, into the industry’s psyche and, more importantly, into potential buyers’ subconscious.

Another aspect of pervasiveness is that whenever a new Android ad is shown, Android’s graphical user interface becomes more familiar to viewers. It’s clear now that pretty much anyone who has watched a few of these commercials will feel some sense of comfort and familiarity with Android’s user interface, even though they might never have used an Android phone directly.

And of course, the main advantage behind pervasiveness is product placement. I’m not talking about direct ads, like the one I mentioned above, but direct inclusion in popular shows such as 24 or even Survivor, just last week.

The amount of daily exposure that Android receives through all these means is nothing short of staggering, and the fact that it’s happening in such a discreet and continuous way makes it all the more effective.

A few years ago, I heard the following anecdote:

A Nokia executive was in an elevator with a dozen other people. During the ride up, several cell phones started ringing and all of them played Nokia’s famous default ring tone. The Nokia executive smiled and said “Ah… the sound of market share”.

It’s becoming increasingly clear to me that you will soon be able to replace the name “Nokia” in the anecdote above with “Android”.

A cloud? More like a black hole

I have been wanting to put my music in the cloud for a very long time.

For back up reasons, of course. After having spent hours ripping your entire collection to bits, you don’t really feel like doing it again any time soon (although the urge occasionally comes to me to do another pass with a better encoding scheme). However, it’s not very difficult to cover this back up aspect since all you need is an external hard drive, an ISP or simply more than one computer.

The appeal of making your entire music collection available from anywhere at any time is very hard to resist, though, but until recently, I have never really found any of the available services very compelling. Add to this the persistent rumors that Google and Apple will soon be offering such a service and I found myself squarely in the “wait and see” camp because I was pretty sure I would end up using one of these two services.

And then out of the blue, Amazon beats them both to the punch with its Music Cloud.

I was still conflicted, though, because I really wanted to wait to see what Google and Apple will come up with, but I recently decided to take the jump and try out Amazon. So I subscribed.

This post is not about Amazon, though, and since I’ve only been using their service for a week, I don’t have much to say about it, but here is a quick review.

My collection takes up about 40Gb,so I chose the $50 plan which gives me 50Gb on Amazon’s servers. The upload of my entire collection took about fourteen hours and went by seamlessly. The Android Amazon application is the same I already had on my phone, except that the previous version was only used to buy MP3’s. This new version also includes the player and it’s been working fine. It allows you to download songs only on wifi if you want to (an important detail, because even though your carrier is pretending that you have unlimited data, you most likely do not) and the preferences mention some “caching” abilities, although I haven’t had the time to investigate exactly what that means (does this cache contain the real MP3 files that I can access from other applications or did they scramble them?).

This move got me to step back and look at the bigger picture, and I realized the the Cloud had been slowly but steadily sneaking up on me these past years. I never really noticed it and I never made the conscious decision to use it, but the amount of information that I have stored on some random servers is pretty impressive:

  • My personal email, obviously. I switched to Gmail in 2004 and everything is stored on the Gmail servers since then. Sadly, all the emails that I exchanged since I started using email (around 1988) are lost. At some point, I was running my own sendmail server on my personal computer, needless to say that I will never, ever do that again (nor should you if you value your time, your sanity and the integrity of your email address).
  • My contacts. I initially stored them in my Gmail account but there was little point in it since I could hardly make use of anything but the email address field. Android changed everything: suddenly, all my Cloud contacts could automatically show up on my phone. Even now, almost three years after Android officially appeared, I’m still stunned that I can add a contact on my computer and have it automatically appear on my phone a few minutes later without having to ever connect my phone to anything.
  • My pictures and videos. Picasa contains all my pictures, but obviously, there are plenty of other quality services out there (SmugMug and flickr to name a few). I have done a better job at retrofitting pre-digital era material here: I scanned a lot of older pictures and uploaded them to Picasa as well. The oldest pictures I have there date from 1998. I wish I had a few older ones as well.
  • Various documents, sensitive or not. Google docs is great for this, both for my own documents but also for documents that I share with others. I’m never far away from a login name on some obscure web site, an account number or a driver’s license number.
  • Applications and programs. The record is more mixed in this area and so far, only my video games are being accurately tracked by Steam. Whenever I install a new computer, reinstalling these games is usually a matter of a few clicks and a few hours of download. Steam is awesome and it’s the future of digital distribution, but I haven’t yet found a service that will extend this to any application that I want to reinstall on my computers.
  • Various files and odds and ends. These are files that I use for development or other activities (configurations, preferences, etc…). I don’t have a good solution for these so I usually just copy them myself on the host maintained by my ISP. Dropbox has recently become a good solution to share files I need everywhere.
  • I’ve never been a big DVD buyer and this trend is not likely to change much now that so much movie content is available online. Netflix, Hulu, Amazon, iTunes and now even YouTube, movies are everywhere, and even if their access is sometimes a bit crippled compared to a good old physical disk (lower resolution due to streaming or availability delayed compared to retail), there is no doubt that these hurdles will soon completely disappear.

The consumer cloud is here and it will slowly continue to improve and swallow our lives, because the convenience is just too hard to pass up.