Standard Libraries

Please excuse me for a moment while I lament a time when the beer was cold, the weather was warm and ifood.com.au returned a 200. At iFood we took pride in perfecting the most minute details and solving the hardest challenges. Ah… that was the life.

One warm summers day, we (and by we, I mean Myles) looked at Ruby's File class and created what seemed like a more “Ruby-ish” (object orientated) approach to file manipulation: Fancypath. Every instance of File.read and File.exist? (there were many, as data was persisted to Git) was happily replaced with syntax like:

(__FILE__.to_path.dirname/'my_file').touch
tmpdir.join('fax','request').exists?

This seemed to make much more sense than the standard File class.

Soon after doing this however, we started getting “random” bugs in the code. Fancypath was subclassed from Pathname, an excellent but little known component of Ruby's standard library. As it turns out some of Pathname's methods explicitly return Pathname instances rather than using the more generic self.class.new. This meant that our Fancypath objects would return Pathname objects only some of the time and thereby made chaining unpredictable and difficult.

Pathname's default behaviour was quickly overridden but the real problem here wasn't Pathname. Before I tell you what it really was I'm going to offer some more evidence of its symptoms.

Almost two years ago at a Rubinius Sprint I paired with Matt Allen to specify some of Ruby's Matrix library. While writing our specs, we found that in certain cases the library actually returned incorrect results. Yeah, bits of Ruby don't work. Even though we found and subsequently fixed the problem, all we could do was test the negative and hope that our fix made it into MRI. Bummer.

The standard libraries that ship with languages have always struck me as kind of odd. Who maintains the quality of these libraries? What happens if that library has a bug? Do updates only get issued with new versions of the core language? And who determines that a library is a standard in the first place? The real problem in the two previous examples is that Ruby ships with an entirely static set of libraries that are not given the chance to change.

So how do we fix this? Considering that you are a language designer like Matz, let's explore the simplest possible option: remove all bundled libraries. Ship your language with only enough to make it Turing complete and let users implement everything on top of that. Rubinius with only the C++ VM. This would result in a lean language but it would be hell to do anything practical with. Let's move on.

Gems seem to work well as a way of distributing code. What if, instead of writing a complete standard library, you just wrote an awesome package manager. Ship the core of your language then have users install file or install math. Core language releases would be orthogonal to the standard library releases. Ideally it would be very easy for the community to fork and contribute changes which would let you focus on the language itself. It seems as though perhaps the biggest hurdle in language adoption today is not the ideas or implementation behind a languages, but the depth and quality of its ecosystem (this is perhaps the biggest selling point for the JVM).

To do this properly, a version number is essential when loading any file from the load path:

# Passing a symbol denotes the lookup is via the load path.
require :fancypath, '0.5.13'

Though this is valid Ruby syntax (and could be easily implemented) this is beside the point. A language needs to harness keen coders like Myles and I at iFood, right from the get-go in order to shape the best and most relevant library of code. Language designers need to focus on what they are good at, pick 10% of that, then delegate the rest.