Quibbles and Bits

A Brief History of Life, The Universe, and Everything

Episode IV – A New Hope.

I don’t know of anybody who has successfully made the transition from CDs to computer-based audio, and abandoned it to go back to CDs again.  Once your music collection is safely tucked away on Hard Disk, the ability to navigate through it, to prepare playlists and collections, to browse intelligently – even to control it remotely using a mobile device such as an iPad or an iPhone – massively enriches the experience.  Even with a relatively mundane piece of software such as iTunes.

My friend ‘Richard H’ has something exceeding 3,000 CDs in his collection.  They live in a selection of shelving units and cupboards that dot his listening room.  Normally, Richard knows pretty much which CDs sit on which shelf, but occasionally some are hard to track down (particularly if I’m the person who last put it away…).  Extracting full value from that collection involves not only remembering everything you ever bought, and knowing exactly where each CD can be found, but also having a good memory for what tracks are on every one of those disks.  I’m sure many of you will identify with that.  It’s what we’ve all gotten used to.

More alarmingly, my sister Barbara once worked for a NPR radio station, WKSU, which is one of the biggest classical music stations in the world.  Their music collection comprises TENS OF THOUSANDS of disks, and their ability to function as a station relies to a great degree on a small number of individuals who work there knowing how to find every last piece of music in the collection.  It is a nightmare of a task, and I have no idea how they can make it work.  The thing is, with classical music, how do you organize a library of thousands of CDs with the sole assistance of a BIG shelving unit?  Do you do it by composer, by musical style, by period, by performer, by record label … or do you just stack ’em up in the order you bought ’em?  There is no obvious optimum solution.  Particularly since, with classical music, a single CD can contain works by different composers, in different musical styles, of different periods, by different performers, and so forth.

Now, if only you could rip those CD libraries into collections of computer files there is an immediate, and very obvious solution.  All that information is just data, and computers handle data very, very well.  The challenge, therefore, is to get all that valuable data off the discs and into the computer.  But this is where the problems start – because the data isn’t actually on the CDs in the first place.

All of the information that describes or pertains to the music on a CD is termed “metadata”.  It is delivered to the customer on the jewel case artwork, and in the enclosed booklet – but none of it is encoded on the disk itself.  Back when the CD format was devised nearly 40 years ago, the concept of wanting to extract such information from the disk did not occur to anyone, and so nobody thought to standardize any method for putting it there.  Then, in the mid-1990’s, following the emergence of the Personal Computer and the CD-ROM, when a method for combining audio and data onto the same disc was devised, there was no interest within the industry in establishing a standard for using it to store metadata.  So it never happened.

What did happen was what always happens when an arrogant industry fails to address the needs of their customers – clever geeks step in and engineer a solution of their own.  In this case it was called MP3.  Techies realized that they could actually play their music on their computers, if only they could get the music in there in the first place.  The trouble was, music files were so darned HUGE that you couldn’t fit many on the size of hard drives that were available at the time.  It is easy to forget that back in those days the capacity of a CD comfortably exceeded the capacity of most computer hard drives – my own first CD-ROM drive came on a PC with a 60MB Hard Disk (that’s MB, not GB!).  It was necessary to do something to get the size of the files down, and that something was the MP3 format.

Using MP3 it soon became possible to collect a fair-sized number of music files on your computer and play them using some custom software (who remembers Winamp?).  Of course, if you wanted to be able to properly manage the new music collection on your computer – or even just identify which tracks were which – you were going to need some of that “metadata” that I described.  So in 1996 a geek named Eric Kemp developed the ID3 metadata tagging system, which was a way to embed metadata into the same files that contained the music.  MP3 became a file format that would store not only the music, but also the information that describes the music.  It was a revolutionary development, one to which the music industry responded with various enlightened approaches including refusing to accept it, pretending it didn’t exist, and trying to ban it.

With the music industry standing off to one side with its head in the sand, the next thing the geeks did was to come up with huge on-line databases which “cloud-sourced” (as we would describe the activity today) all of this metadata, along with interfaces that individual users could use to interact with it.  Users could insert a CD into their computer and rip the audio data to a set of MP3 files.  Then, software would analyze the CD, and by interrogating the databases it would identify it, locate all of its metadata, and – bingo! – automatically insert it into the resultant audio files as part of the ripping process.  If it couldn’t identify the CD, you would enter the data manually, and the software would upload it to the database so the next person to rip that CD would be able to use it.  It was a revolution that happened without the participation – or even the support – of the industry.

The new millennium marked the “end of the beginning” (if I may channel Churchill), when Hard Disks became widely available that were big enough to hold the contents of a large number of CDs, and in response the geeks started developing alternative formats to MP3 which could store the music in its original (i.e. lossless) form – the FLAC file format is by far the most popular – thereby preserving intact all of the musical information.  These new formats would also support the new high definition audio standards that were emerging at the same time.  And the audio hardware industry – unlike the music industry – would enthusiastically leap on board with new products to allow users to integrate these new developments into their music systems.  And they would do so at all levels of the quality spectrum.  Today, if you wish to do so, you can download music in the DSD256 format, where individual tracks can consume an incredible 1GB of disk space!

Thus, driven by an enthusiastic, responsive, and customer-centric audio hardware industry, the computer-based audio paradigm reached its first level of practical maturity.  The record industry at first refused staunchly to participate, but could not afford to ignore it indefinitely.  But by the time they finally got on board with downloading as a legitimate mainstream sales & marketing channel, they could no longer hope to drive – or even, frankly, influence – its de facto standards, which continue to evolve pretty much independently, for better or for worse.  It is also why, at some point in time, Apple will step in and eat their lunch, and their supper, and will even clear out their wine cellars.  Yes, you read it here!  But for the time being, we consumers have gotten way too comfortable with the notion that we control our own destiny, where in fact we are ripe for a major shake-up.