The Glass Ceiling of Software Design

Brief Recap

In part 1 of this series, I examined what I felt was a central contradiction in how we use computers: we expect them to save us from rote, tedious, repetitive, and error-prone work, and yet designing computer software can be a rote, tedious, repetitive, and error-prone task.

By manually applying programming and UX patterns to the design of software, we can guarantee a certain level of consistency from one app to the next. Nevertheless, by mis-applying existing patterns, disregarding best practices or making other errors when crafting computer software, we unwittingly allow anti-patterns to destroy this hard-fought consistency.

This is to say, computer code is ephemeral, and software only works until it’s broken again.

Owning Music

From the time I discovered the existence of music in the MP3 format, I became infatuated with the idea of collecting and cataloguing music. Anytime I wanted to add to my library, I’d search for a song or album I enjoyed, download it to my computer, and drop it into my music library. I would spend a lot of time carefully crafting the metadata in my music collection. This meant ensuring that all tracks, album and artists names were properly spelled and capitalized. I would make sure that every song was placed in the context of an album, with the track numbers filled in and album artwork attached. Recording years, genres, lyrics, and composers were also painstakingly researched and slotted into the appropriate fields.

The iTunes Metadata Editor

The iTunes Metadata Editor

I spent hours doing this. The idea of owning a well-curated and -catalogued music library very much appealed to me — as it does to a lot of music collectors. And hey, what can I say, I’m the son of a librarian. I’ve written about metadata and cataloguing elsewhere on this blog, with particular reference to searching for music.

At any moment, a stranger could look at my music library and get a snapshot of what artists I enjoyed listening to, what music styles and years I was drawn to, what my favourite tracks were, and so on. Why was this important to me? In some senses, it was functional, in that having a well-annotated music library could allow me to construct specific playlists, to suit my moods and tastes. In other senses, it was purely aesthetic.

iTunes allowed me to own, maintain and be proud of my personal radio station, programmed expressly to cater to my tastes.

Along Came Spotify

I had heard a healthy amount of buzz about Spotify from some Swedish colleagues of mine, and in 2010, on a trip to Sweden, I finally got the opportunity to try out the desktop app first-hand.

The first thing I noticed was that the interface bore many resemblances to Apple’s iTunes. In a single window, one could find their playlists on the left, their music library in the main pane, and search, and filtering options at the top.

There were some variations, but by-and-large, iTunes and Spotify were very similar products. The biggest difference, of course, being that Spotify is an online product which streams music from a remote library. This means that, unlike iTunes, it is impossible to have a global look at the entirety of the music library, due to its sheer size (Spotify’s global catalogue ranges into the 20+ millions).

Indeed, with Spotify, it was necessary to have search capabilities in order to locate the song, album, or artist you wanted, and then play it back. iTunes relied mainly on a browsing interface.

Another direct consequence of this architectural difference between both products is that Spotify provides read-only access to its song database, and in my case, that meant that curating the music library as thoroughly as I had previously done was impossible. If I spotted a typo in a track name, for instance, I would have to flag the track as having inaccurate information, in order to notify a Spotify curator to fix it. There was a shift, from owning the music and the data, to simply having access to it. In Spotify, the editorial opportunities available were limited to creating playlists.

Owning Features

Apart from Spotify and iTunes, there are countless music players of various architectural styles, and naturally, the one base trait they must all share is the ability to play music. Beyond this, each one offers a grab-bag of features to support the central task: metadata editors, smart playlists, visualizers, integrated lyric searching, etc.

I take issue with the current practices of developing software because, as I referred to in the first part, it is a labour-intensive process which requires a certain knowledge of the inner workings of computers. Not everyone is inclined to learn the discipline of computer programming, and so programmers and UX designers must step into the shoes of an end-user in order to anticipate and dictate which features are included, and which aren’t.

For example, when I find a song that is particularly appealing to me, I tend to obsessively binge-listen to it. (It can get pretty bad. I remember playing and rewinding “The Sign” on my Ace of Base cassette album at least 20 times in one sitting. But I digress.) Computers remove the need to rewind songs in order to listen to them again. Most music players will even include a handy Repeat One feature, for people like me.

Consider this: here is a feature that I know and enjoy, and also a feature which is not particularly difficult for a computer programmer to implement. To describe the logic behind the feature is dead easy: when the song finishes, play it again.

I enjoyed this feature when using Apple’s iTunes, but was dismayed to find it absent in Spotify. Luckily for me, the company introduced it to their desktop player in February of 2014, and announced it with a blog post. “You asked and we listened!” they announce.

Unfortunately for me, as of the time of this writing, this feature is still unavailable in the Android app.

I believe this perfectly illustrates another intrinsic absurdity in the process of manual software development: a feature that I use and enjoy, and which is incredibly easy to describe and implement, is not necessarily available to me. I must rely on the software maker to implement and maintain it. Apple and Spotify, for all of their talent, are limited in what they can design and implement, and so feature requests that shout the loudest get answered the fastest.

Conversely, because of Spotify’s and iTunes’ auto-updating features, there exists the possibility that the features I know and enjoy won’t be there tomorrow. Repeat One exists in Spotify on my Mac, but for how long?

Software ecosystems are in a state of constant flux. Features which were previously available or known to you might be removed, modified, or broken, with each subsequent software update. New features and paradigms are introduced all the time, as products evolve, grow, and shift their focus.

A handy but little-known feature might be removed from one version of an application to the next. Conversely, an incredibly-useful but obscure feature that would make your job ten times easier might never get built, because the software’s developer has other priorities.

These are human-motivated changes, but what about unmotivated ones, such as bugs and anti-patterns? While we’re on the subject of iTunes, have you heard about the installer for iTunes 2 which could potentially erase your whole hard drive? And not to play favourites, Spotify recently found a severe bug in their software which required every Android user to manually upgrade the Spotify app, and re-download all their previously-downloaded music. This is not a dig at either company — programming software is very tricky business. Even with the incredibly bright minds at both companies, well, shit happens.

Owning Your Computer

If you want to teach people a new way of thinking, don’t bother trying to teach them. Instead, give them a tool, the use of which will lead to new ways of thinking.

– Buckminster Fuller

I think there’s a much better way to make software, and I alluded to it in the previous article. By using computers to write computer programs for us, we can do away with many of the human errors which result in features breaking.

By delegating the rote, tedious, and error-prone job of programming to a machine (which is why we built the machine in the first place), we can focus on using our intrinsically-human gifts of creativity to design software faster, much cheaper, and without the resource bottleneck that exists today.

In much the same way as iTunes allowed me to own and curate my own music library, I’m looking to have the same experience on my computer platform at large. I can own the physical boxes that run the software, but as far as the software itself goes, I mostly have to trust that other people will do things that make my computer useful and pleasing to me. What if there were a better way?

Imagine being able to build all of your own software. In the case of a music player, you could start by having a list of hundreds of different features in front of you, each with its own checkbox. You’d go through the list, checking the boxes of features you wanted, and leaving unchecked those features you didn’t care for. At the end of the form, you’d find a button which, when pressed, would churn out a bespoke music player, tailored precisely for your tastes, having exactly the capabilities you needed.

Imagine, as well, being able to describe a feature, e.g. “the ability to listen to a single song on loop,” and have that be all of the effort required for you to have that feature built into your bespoke music player.

Going beyond that, imagine the capacity to describe a feature that no one has ever heard of or implemented before, and have it be built and crafted into your music player, e.g. “Always repeat songs by Tom Waits twice, before moving on in the playlist.”

In order to have such a software ecosystem, we need to bridge the gap between human description of features, and their technical implementation. We do this by isolating and cataloguing patterns of computer software design, and their meta-patterns of assembly. How does a feature like the Repeat-Waits break down into smaller components, (i.e. “repeat twice” and “identify Tom Waits songs”), and how do those components fit together in a meaningful way?

The translation of meaning, from human description to software implementation, is key to this new way of programming, and this is where I think the ideas of semantically linked data have a lot to offer.

The Human Side of Computing

Every day, and especially apparent from my vantage point in the Silicon Valley, more and more people are spending more and more time online. Smartphones and tablets have made computers portable like never before, to the point where connecting to the internet while riding a train is a pretty ubiquitous sight.

Undoubtedly, the human race is forging an ever-stronger attachment to computers. As technology brings itself ever-closer to the social, creative, educational and bureaucratic spheres of life (to name the most salient ones), the interface between humans and machines takes on a central role in the design of technology. User Experience (UX) designers study this very problem: how can we fashion interfaces that allow computers to become virtual extensions of ourselves; of our thinking and of our bodies?

Touch screens, for instance, have proven to be a natural and extremely intuitive way for people to manipulate on-screen objects like photos, web pages and game sprites. I constantly marvel at how very young kids seem to innately understand multitouch interfaces like those on tablets.

This is the world we live in: mothers giving their children iPads to keep them entertained and engaged during an otherwise-boring 4 hour flight.

And yet, if we look beyond computers-as-entertainment, and begin peering into the world of computers-as-tools, we begin uncovering a much different landscape.

Let’s start with what I think is a very reasonable statement: Computers are such powerful and useful tools to us because they allow us to accomplish with relative ease what might be incredibly tedious, error-prone and difficult to do without them.

To illustrate, I cite the example of William Shanks (1812-1882), who devoted many years of his life to meticulously working out the first 707 trailing decimal places of pi. It was monotonous work. He would spend his mornings generating new numbers and his afternoons verifying his morning’s work. In 1946, it was discovered that he had made an arithmetical miscalculation at the 528th position, causing the remainder of his work to be inaccurate and therefore worthless.

Today, I could calculate the first million digits of pi without even taking a single bathroom break. Or, if I really need to go, I could probably even Google it. Computers are exceptionally good at tedious, repetitive work.

The Cult of the Programmer

We therefore hold ever-higher expectations of computers. As they increasingly become extensions of our physical and social selves, we have come to expect and appreciate the newfound ease with which we can entertain ourselves, connect with others all around the world, and facilitate our daily lives.

We also expect powerful microprocessors to make astounding numbers of calculations to facilitate encryption, audio and video compression, web searching, gaming and countless other wonders of software engineering.

All along the spectrum, from the processor on one end, right out to the user on the other, we have the art and science of computer design.

UX designers tend to build closer to the human end of the spectrum, while programmers (and other highly-technical workers) build at the other end, and are much more involved with the nitty-gritty technical details of making the computer do what you ask of it.

The process of software design is all about patterns. They exist as a basic unit for writing computer code, as well as a guiding principle when designing the user interface. A pattern refers to a tried and tested way of implementing an idea. While the details of the implementation may change (e.g. the programming language in software programming or the screen size in UX design), the underlying structure behind it is based on a repeatable convention.

Patterns can exist at all scales, from the macro (e.g. the user flow through an app) to the micro (e.g. two lines of computer code), and what exactly constitutes a pattern could be the subject of debate.

Nevertheless, software design could be seen as the process of assembling a collection of software and UX patterns as building blocks, and like Lego, assembling them in an interlocked fashion to form a greater whole. A skilled programmer can then refine and modify the patterns to the degree needed to get the precise desired result. (Unlike with Lego, where taking out a piece of sandpaper and actually rounding out the edges will just get you many stern looks and chastisement from your playmates. Especially Becky.)

To my mind, we’ve already arrived at a certain contradiction: we have computers in order to save us from tedious, repetitive and error-prone work, and yet the very process of software design is at least repetitive and error-prone, and at times, tedious.

By repetitive, I mean that the same work must be done over and over again. That is, in order to accomplish a similar result, the same efforts must be repeated time and again. While there exist patterns and best practices that emerge from developing a software domain, regressions are generally commonplace.

Consider the case of web servers, which respond to HTTP requests — the basis for delivering web pages from a disk drive somewhere to your iPad screen. Web standards and best practices have been evolving constantly for over 20 years, to bring us the rich web we enjoy today.

If I wanted to build a piece of software with its own web server, and I wanted to do so from scratch, it would be a considerable task, much bigger than back in 1991, when the web was just beginning. To make it fully interoperable with today’s web, it would have to include support for cacheing, virtual hosts, streaming, reverse proxies, encryption and a whole host of other enhancements to the original web specifications. And still, in the process of building my software, it is possible that I apply the wrong code pattern, or that I simply make a typo which breaks basic web functionality. If I’m really unlucky, the bug will make its way out into the released software, and more than 20 years after the inception of the web, it is still possible to run across a web server which doesn’t implement the basic protocol correctly.

The recent Apple SSL bug is an excellent example of this very phenomenon: almost 20 years and who-knows-how-many successful implementations of SSL later, it is still possible to release a version which is completely broken in a fundamental way. Worse, the error looks like it may have been the result of a sloppy copy-paste operation on behalf of one of their programmers. Copy-pasting code is a pretty good signal that a task is rote, and in this case, that resulted in a very large and very costly software flaw.

I don’t mean to attack Apple over the flaw, but it provides a good case study into the problematic aspects of programming. It also points to the very ephemeral nature of software code, and how bugs are only fixed until they’re not.

The recent Heartbleed SSL vulnerability runs into similar themes. At its core, the vulnerability was due to a specific type of bug which has been found and fixed so many times it has been given a name: missing bounds check. (This is what could be referred to as an anti-pattern – a known problematic pattern).

Hindsight is 20/20, and it’s tempting to think that every now and then, one or two anti-patterns make their way through the software testing process and wreak a lot of havoc in the real world for a limited amount of time, and that once found, the threat is neutralized. Poul-Henning Kamp, a respected software developer and columnist for ACM Queue estimates that the computer you are reading this on may contain up to 1000 more such vulnerabilities.

It’s also possible that some, most or all of these bugs are caused by well-known anti-patterns: buffer overflows, SQL injections, OS command injections, etc. (The SANS Institute maintains a list of the top 25 most dangerous anti-patterns). The underlying issues have been so well studied that in many cases, applying the fixes is in itself a rote activity.

I am left feeling like designing software is somewhat like William Shanks’ endeavour: while we no longer do the calculations for pi by hand, we create the algorithms to calculate the digits of pi by hand, and we keep making a mistake on the 528th line.

Given the earlier contradiction in how we expect and use computers, it begs the question: Could programming be better accomplished by computers?

(Continued in part 2)

Release Notes – January 12th, 2013

This latest release of the Exxis Semantic Web Browser has been a long time in the making, and that’s because it has been entirely rewritten for speed and flexibility. On the front end, all HTML and Javascript code has been cleaned up and optimized. On the back end (i.e. under the hood), a massive rewrite of the architecture makes the core software more powerful than ever.

Now that I have reached a major milestone on the software side of the application, I will now turn my attention to the visualization of the semantic information presented, to make it much more intuitive. Lots of exciting developments are forthcoming, stay tuned!

  • Lightning-fast searches!: I managed to get search times down from over 30 seconds to less than 200 milliseconds, quite the dramatic improvement! This is fast enough for instant search! Try it! Just type a term you’d like to know more about, and without even pressing return, you’ll get a list of search results.
  • Instant Search

  • Complete architectural redesign: Pretty much everything you see has been redesigned, as well as everything under the hood. This release represents a major architectural overhaul of the core software, to support plugins. This means that it should be much easier for me and others to develop new visualizations and reasoners for semantic information. In other words, this release represents the beginning of the Exxis Semantic platform! Stay tuned!
  • Back button works!: This is surprisingly tricky to get right, in the world of web applications, but the back and forward browser buttons should work as expected.
  • No support for Internet Explorer: I’ve decided to drop support for Internet Explorer, as version 8 is very lacking in its HTML5 support. You’ll get a much better experience when viewing the site on Firefox or Chrome.

Release Notes – October 11th, 2012

This post is used to outline the changes brought to the October 11th, 2012 release of the Exxis Semantic Web Browser. Try it now, and be sure to leave feedback!

  • No more pizza!: I have done away with the basic Pizza ontology that I was using for testing, and have replaced it with the OpenCyc knowledgebase. The browser can now display information on more diverse topics such as, for example, Princeton University, the Montreal Canadiens, or, if you still prefer, pizza.
  • Search: Since the general-knowledge dataset provided by OpenCyc is much larger than the pizza dataset, it no longer made sense to list every object in the left sidebar. A search interface replaced the object listing.
  • Asynchronous AJAX: The presentation layer is now asynchronously connected to the view layer.
  • New display: I have made lots of changes to the overall user experience, with many more changes to come.
  • Separation of data and object attributes: Data attributes (e.g. strings, integers) have been moved out of the graph, into the “Properties” pane in the sidebar. The graph now only displays object properties.

Release Notes – September 21st, 2012

This post is used to outline the changes brought to the September 21st, 2012 release of the Exxis Semantic Web Browser. Try it now, and be sure to leave feedback!

  • Dynamic Navigation!: Nodes can now be clicked, to expand them and re-focus the graph. When clicking on a node, its label will first turn orange, to indicate that it is loading. Once the node has loaded, the label will turn green, and any additional nodes will be added to the graph. This gives way to a much more dynamic way to browse the graph. Try, for instance, viewing a node like DomainConcept, and clicking on nodes to explore the dataset from there.

Release Notes – September 9th, 2012

This post is used to outline the changes brought to the September 9th, 2012 release of the Exxis Semantic Web Browser. Try it now, and be sure to leave feedback!

  • Many small bugfixes: I am constantly finding and correcting software bugs here-and-there. Last week, there were 2 server outages related to odd problems with both the MySQL and RDF libraries I’m using. In this release, I have found two such bugs and implemented workarounds.
  • Node clustering: Nodes are now clustered when they share the same predicate, to simplify the visual display by reducing the number of lines.

    Before – No clusters were being drawn.

    After – A cluster of 5 elements was detected. The ‘hasTopping ~’ label is now drawn once, instead of 5 separate times.

Release Notes – September 1st, 2012

This post is used to outline the changes brought to the September 1st, 2012 release of the Exxis Semantic Web Browser. Try it now, and be sure to leave feedback!

  • MySQL backend integration: The previous version relied on an SQLite database, which was not optimal for anything bigger than a very small semantic database. This new database integration adds robustness and speed. (I will shortly add support for Postgres, as I hear it is much more optimized than MySQL — I just chose MySQL because I know it well.)
  • More consistent graph display: The focused node now appears at the centre of the canvas. Superclasses are displayed above. Subclasses and instances are displayed below. Incoming and outgoing nodes, which are not related to class hierarchy are drawn to the left and right, respectively.



  • Fewer nodes: A lot of the feedback I got was along the lines of, “I don’t get it”. I’ve therefore filtered out many of the predicates that don’t intuitively mean anything to most people (e.g. disjointWith, first, range, domain, onProperty etc.). I’ve also filtered out most of the Restriction objects from the graph. These connections will re-appear in later releases, presented in more functional ways.