The Exxis Semantic Blog

Bridging the gap between humans and computers

2012-01-25 15.37.46

When Does My Computer Get Here?

In 2012, I ordered a computer online for the first time. Every computer I had purchased before that one had been studied, manipulated in-store, revisited, coveted and eventually and with great enthusiasm, purchased.

It’s this enthusiasm to own a shiny new beast of a machine that led me to Apple’s online store, where not only did I pick the largest base model, but then talked myself into upgrading every single component inside it. I added, among other things, a larger hard drive, a secondary solid state drive (SSD), a beefier processor and more memory. (I could almost hear Tim “The Tool Man” Taylor grunt in approval as I made these decisions. “More power!”)

The in-person experience of taking physical possession of a computer had been replaced by an online storefront proxy, but my enthusiasm had not dampened in comparison with every earlier computer-purchasing experience. If anything, it was more palpable. “When does my computer get here?”, I’d repeatedly ask myself in the following days, until the moment when the large brown UPS box was delivered and definitively in my living room.

That was three years ago. I am writing this blog post on The Beast now, and I’m happy to say that it has been serving me well ever since its unboxing.

Well… there is one thing that never worked properly. The Wi-Fi circuitry inside the computer seems to have a problem. Or maybe it’s the software that handles that circuitry. In essence, the machine was billed as being able to connect to Wi-Fi networks at three separate speeds: A/B (slow), G (medium), and N (fast). I’ve been able to successfully establish and maintain connections in the first two modes, but when it comes to N, for some reason there are intermittent failures, like blackouts, in the signal. Sometimes the connection will be fine for 15 minutes, and, without warning or obvious reason, my computer will be unable to connect to the network for the next 2-3 minutes. Then, the signal will come back for 30 seconds, before going down for another 2 minutes. Then up for 8 minutes, just enough to lull you into a false sense of security, only to go offline for the next 40 seconds. And so on, and so on, and so on.

It’s a frustrating problem to have, but more than that, it feels like The Beast has a limp in one of its legs, making it unable to… uh… run at full speed and… uh… catch data gazelles?

The point is, I spent hours working on this one problem. I tried different routers. I tried placing the router antennas in different positions around my house. I collected hundreds of thousands of metrics on my tiny 1-device home network, in search for a pattern, in search for an Aha! moment that, sadly, never came.

I even corresponded briefly with an Apple engineer. He reached out to me when I joined an online discussion of users having similar problems as mine. I sent him all of the data and analysis I had collected and then… nothing.

Sadly, we never found the problem or the fix and The Beast still limps along at “G” speeds, rather than the “N” speeds which were promised on the outside of the box.

The Illusion of Progress

From that time, three years ago, when I unboxed my new desktop computer, the proliferation of smartphones and new, ever-more-powerful computers has not slowed a bit. However, this continuous integration of software and internet technologies in everyday life is anything but smooth.

To illustrate how rocky is this road, I’d like to present a sketch of some of the more dubious human/computer interactions I might run into in a typical day of working in the Silicon Valley:

7:20am: One of the first things I do after waking up is to look at my weather app, to get ideas as to what I should wear today. It looks like it’s going to be a very comfortable 18°C by 3pm this afternoon. That sounds like T-shirt weather, but if you’re not careful, you might miss the part where it says, in much smaller print, that winds will be gusting about at a mean 34 km/h today, just like they did yesterday. Make no mistake about it, it’s jacket weather. It has never really made sense to me why weather apps generally choose to highlight the wrong information, like showing the actual outdoor temperature (18°C) in large print, while the more useful “it feels like this” temperature requires a few extra taps to find.

7:49am: Aside from finding out what to wear, the other daily question I need to answer is “How do I want to get to the office today?” I can either take the train, or I can drive my car. I only want to drive if there are no events at San Francisco’s AT&T Park, which is only 2 blocks from my office. If there’s an event today, not only do parking prices in the area quintuple (from $12 to $60+), but traffic can be a nightmare. For this reason, I often need to research and answer the question, “Is there an event today at AT&T Park?” As a consequence of me running this search routinely, my phone has become obsessed with showing me the scores of Giants baseball games, at home in SF and away. My interest in AT&T Park is misinterpreted by various recommendation algorithms as being an interest in the Giants.

8:31am: I decided to take the train. I’m standing on the platform. The digital readout announces that the train is “4 minutes late” and yet, here it is, pulling into the station at exactly the scheduled time. Strange.

9:29am: I arrive in the city, and I see that the monitor that displays the schedule for the next few trains is offline, as it has been for about three weeks now. I don’t know when the next train comes, but I do know that Windows Defender needs to be updated.

CalTrain Schedule Monitor

9:39 am: I arrive at the office. It’s Monday morning, and since my work computer has been on all weekend, it has accumulated a number of messages that have popped up to alert me of various things I ostensibly care about:

Update HipChat!

Screen Shot 2014-04-25 at 9.27.02 AMUpdates Available

It generally stands to reason that new versions of software are better than old versions. For instance, in the updates proposed above, you’ll notice that the new versions of “HipChat” and “iTerm” (two pieces of software I use many, many, many times a day) contain new fixes for existing bugs. It’s tempting to think that this auto-update behaviour is continuously improving the stability of my computer every time it proposes an update, and indeed, this is the main narrative behind the auto-update feature: newer is better. This is a misleading, if not patently false, impression. There is no dialogue box that shows you what new bugs are introduced in every new version, and sure enough, that’s exactly how they keep getting onto your computer.

9:44am: It takes a few minutes for the software updates to complete their installation, so I take advantage of this brief morning intermission to grab a cup of coffee. Crushing it!

9:45am: One of the slightly annoying things about these updates is that they sometimes ask for “permissions”. You see, certain parts of your computer that contain sensitive or private information are protected by software “gates” that require the user to explicitly give permission for the sensitive information to be shared with the requesting application. In this case, CoreServicesUIAgent is asking for permission to access my calendar:

CoreServicesUI Would Like To Use MY Calendar?

I don’t know what CoreServicesUIAgent is, but my intuition tells me that with a name like that, it’s some kind of system program that, for some reason, needs to access my calendar. I don’t know what that reason is, but I do know that refusing to grant this permission would mean that two pieces of my operating system wouldn’t be able to talk to each other. I know from past experience that this can lead to unexpected, quirky and seemingly random behaviour in the future. Refusing this permission is tantamount to breaking a pipeline somewhere in the complex machinery of my computer. Is it an important pipeline? That’s very hard to answer, given that I have no idea what CoreServicesUIAgent actually does.

(The permission screen is a security feature, to make sure no virus steals your sensitive information without your knowledge, yet it only really works when the user is informed enough to make a sensible decision. If I were making a virus that aimed to steal your calendar information, I wouldn’t call it Big Bad Mean Virus, because no one would grant the necessary permission. Instead, I would call it CoreServicesUIAgenda, hoping everyone would automatically grant the necessary permission without giving it too much thought. Did you spot the difference? Because most people probably won’t.)

9:48am: After gazing off into the distance and pondering the issue of software permissions for three minutes, I take a peek at what my calendar looks like for today:

Packed Day

I have multiple e-mail accounts where calendars are stored (four on Google, one on the computer and one more on Apple’s iCloud), and because they all try to be helpful by inserting various Canadian and American holidays into my planner, this happens on certain days. As a result of all of this helpfulness, I can’t actually see the meetings I have today without a few more clicks.

10:12 am: I’ve been catching up on e-mails and easing into the workweek, and by now, I’m feeling “warmed up” enough to get into some serious software coding. To do that, I like to put on my headphones, find a good playlist of music that allows me to focus, and let ‘er rip. Only problem is, after about a minute, I notice that most of the music sounds like it’s coming from right side of the room. I know why this is. I open the System Preferences app and see that it’s happened again:

Sound Balance Drifting to the Right

If you look at the Balance slider, you’ll notice that it’s skewed towards the right. I didn’t do that. There’s a long-standing bug in Mac OS X that causes the sound balance to very gradually and subtly drift towards the right channel. Every week or two, I have to go into the sound settings and manually re-center the balance. This has been happening for years and years now. I wonder why Apple hasn’t fixed this yet.

12:35pm: I stepped out to grab a sandwich around the corner. As I’m walking down the street, I spot a beautiful vintage Mustang convertible waiting at the red light. I take out my phone, and snap a picture to send to a co-worker. Alas, my phone freezes just after I’ve captured the picture, and I need to restart it, losing the picture in the process. By the time my phone is back on, the car is long gone. Damn.

12:38pm: As I sit in the busy sandwich shop, waiting for my order to be up, I run through in my head the programming puzzle I’ve been working on all morning. I’ve designed a piece of code that goes to great lengths to make sure that a certain thing only happens a single time. It works flawlessly, except for the minor fact that the thing that’s supposed to happen only once actually happens two ore more times, and I don’t know why. I enjoy working on problems like this, but I’ll admit that spending 20 hours to discover the root cause can be a bit frustrating when all you want to do is tell the computer “Once = good. Twice or more = bad.” I mean, how is that so hard to understand?

3:33pm: In the middle of my afternoon back at work, I find myself waiting for a password reset e-mail to be sent to me, so that I can reset another password I’ve forgotten, and change it to something I’ll forget again soon. I wonder how many times per day and per week do I need to reset the password to some service or account I’ve all but forgotten and in that instant, need? I swear there are mischievous sprites who change my passwords behind my back…

5:05pm: I’m trying to write a short e-mail on my phone, but it’s a frustrating exercise. At this very moment, my phone has decided that it’s time to download and install some software updates. Like my desktop computer, my phone goes through a process of looking for new updates to software that I already have installed.

I don’t have many apps, but the number of updates available on a daily basis adds up quickly. Since downloading and installing software tends to be a processor-intensive activity, my phone does this thing where it freezes for a second or two while I keep typing, before the letters all appear at once, as if trying to catch up. Even more frustrating, every time a new piece of software is installed, my on-screen keyboard disappears for 2 seconds, even if I was right in the middle of a word.

I can’t keep writing my e-mail while these updates are installing, so I can either turn off software updates, write my e-mail, then turn them back on, OR, I can wait it out. Turning off software updates requires me to exit the e-mail app and navigate to the appropriate screen, which would take time and divert my attention from the task at hand. I opt instead to just put my phone down for a minute or two, until the update storm passes. I wish there were a weather app that would tell me how usable my phone will be today.

5:35pm: A friend of mine from France tried calling me on Skype, but I was walking to the train, and thus, was unavailable. She left me a video message, but unfortunately, it consistently freezes about 3 seconds in, when I try to watch it on my phone. I’ll try watching it later on my home computer.

5:47pm: On my earphones, I have a “volume up” and “volume down” button, in order to control the volume of music or phone calls without getting your phone out of your pocket. I notice that in the latest version of the Spotify music player, the “volume down” button doesn’t work. Turning the volume up works, but in order to turn it down, you have to get your phone out of your pocket and adjust the volume from there.

7:30pm: Now home, I check to see if the video message plays any better on my home computer. Before I can try watching it, though, I need to install a software update as well as give permission for Skype to update itself.

Skype Authentication Request


7:31pm: The version of Skype on my iMac plays the video message better than my phone, but not by much. It still freezes at 0:03 mark, but at least I’m able to drag the playhead over to 0:05 and watch the rest of the message, even though it’s glitchy. I wasn’t able to do this on my phone.

8:20pm: I start watching an episode of The Office on Netflix. Here too, I get asked to grant more permissions:

Permission to watch The Office

Jim Halpert wants to know — is it cool if we take this full screen?

My web browser wants to know if I want to allow Netflix to display its videos in fullscreen. Why on earth do I need to grant a permission to show videos full screen?

The answer, it turns out, is that this is another (bad) security feature. If a malicious web app or virus were to make itself fullscreen without my noticing (and without me being required to give explicit permission), then it might be able to fool me. It could, for instance, show me a fake password dialogue box with generic text like “You need to enter your administrator password to continue”. Since I get bombarded all day long with requests for permissions and requests for me to enter my password, this app could easily spoof another one of these screens and trick me into giving it my password.

The solution then, is to ask the user each time a specific web address wants to show its content in fullscreen mode.

I’ve already granted this permission in the past, but the reason it’s asking again is because this time, I’m accessing the site through “” rather than the more common “”. This isn’t a choice I made. I went to “”, and they bounced me over to a secondary server, “www1″. (This is sometimes done in order to distribute web traffic, and not overload one server with too many visitors. There might also be a “www2″, “www3″, etc.)

In either case, my web browser now wants to know if I’m cool with “” going fullscreen. Tomorrow, I might be asked if I’m cool with “” going fullscreen…

The Kind of Day It Has Been

I’ve listed several software issues I encounter on a typical day, and these are by no means all of them. In fact, I would estimate that I experience at least ten times more bugs and software quirks on an average day than I described above. It simply wouldn’t make for a good blog post to exhaustively document some of the more obscure and convoluted issues.

Ultimately, the cause of each and every single one of these issues could be explained away by an expert. For instance, if you were to discuss the issue of the digital readout on the train station that marked the train as being “4 minutes late” even though it was on-time (8:31am), you’d get an explanation which might sound like this: “There are sensors on the track that detect which train is passing at a certain point. These sensors send a signal to a central piece of scheduling software, which estimates the delays. Unfortunately, the sensors at the Mountain View station were made by a different manufacturer than the others, and so they’re not fully compatible with the latest update in the central scheduling software. This means that the information they send is wrong, and because of a bug, they’re off by 23-seconds when they report the times to the central scheduling computer, and this triggers the estimation algorithm to calculate that the train is 4 minutes late.”

The explanations that engineers tend to give for software bugs are always totally logical, and more often than not, immensely unsatisfying. I can see that the train is at the station. Why can a computer not see that very simple and obvious truth too?

Likewise, many of the other issues I’ve detailed above have explanations which are completely reasonable, and completely unsatisfying. In the case of the Netflix/fullscreen permission issue (at 8:20pm), the fact that Netflix might want to put me on their “” server in order to distribute traffic makes sense. The fact that my web browser tries to protect me against software that puts itself into fullscreen without my knowledge makes sense. Taken together, however, the fact that I don’t really know, when I watch Netflix, whether I’ll need to give a permission I’ve given in the past all over again, or not, makes absolutely no sense.

In other words, it makes no sense to me for my computer to ask me to perform seemingly-arbitrary tasks to accomplish things I don’t really care about. I don’t really give a good gosh darn if CoreServicesUIAgent has permission to access my calendar or not. I just want my computer to work.

Why am I being deputized to oversee the continuous updating of software on my computer? Why am I being asked to arbitrate every permission request, as if I don’t have better things to do? Since when is my computer a tool for managing itself, rather than a tool for getting my work done?

It’s been over three years since I ordered my computer online. It mostly works, but every day, I run into new issues and quirks. The “N” wireless capability still doesn’t work, as was promised on the outside of the box, and at this point, I’m pretty sure it never will.

And so, I’m left still asking myself, “When does my computer get here?”

The Glass Ceiling of Software Design

Brief Recap

In part 1 of this series, I examined what I felt was a central contradiction in how we use computers: we expect them to save us from rote, tedious, repetitive, and error-prone work, and yet designing computer software can be a rote, tedious, repetitive, and error-prone task.

By manually applying programming and UX patterns to the design of software, we can guarantee a certain level of consistency from one app to the next. Nevertheless, by mis-applying existing patterns, disregarding best practices or making other errors when crafting computer software, we unwittingly allow anti-patterns to destroy this hard-fought consistency.

This is to say, computer code is ephemeral, and software only works until it’s broken again.

Owning Music

From the time I discovered the existence of music in the MP3 format, I became infatuated with the idea of collecting and cataloguing music. Anytime I wanted to add to my library, I’d search for a song or album I enjoyed, download it to my computer, and drop it into my music library. I would spend a lot of time carefully crafting the metadata in my music collection. This meant ensuring that all tracks, album and artists names were properly spelled and capitalized. I would make sure that every song was placed in the context of an album, with the track numbers filled in and album artwork attached. Recording years, genres, lyrics, and composers were also painstakingly researched and slotted into the appropriate fields.

The iTunes Metadata Editor

The iTunes Metadata Editor

I spent hours doing this. The idea of owning a well-curated and -catalogued music library very much appealed to me — as it does to a lot of music collectors. And hey, what can I say, I’m the son of a librarian. I’ve written about metadata and cataloguing elsewhere on this blog, with particular reference to searching for music.

At any moment, a stranger could look at my music library and get a snapshot of what artists I enjoyed listening to, what music styles and years I was drawn to, what my favourite tracks were, and so on. Why was this important to me? In some senses, it was functional, in that having a well-annotated music library could allow me to construct specific playlists, to suit my moods and tastes. In other senses, it was purely aesthetic.

iTunes allowed me to own, maintain and be proud of my personal radio station, programmed expressly to cater to my tastes.

Along Came Spotify

I had heard a healthy amount of buzz about Spotify from some Swedish colleagues of mine, and in 2010, on a trip to Sweden, I finally got the opportunity to try out the desktop app first-hand.

The first thing I noticed was that the interface bore many resemblances to Apple’s iTunes. In a single window, one could find their playlists on the left, their music library in the main pane, and search, and filtering options at the top.

There were some variations, but by-and-large, iTunes and Spotify were very similar products. The biggest difference, of course, being that Spotify is an online product which streams music from a remote library. This means that, unlike iTunes, it is impossible to have a global look at the entirety of the music library, due to its sheer size (Spotify’s global catalogue ranges into the 20+ millions).

Indeed, with Spotify, it was necessary to have search capabilities in order to locate the song, album, or artist you wanted, and then play it back. iTunes relied mainly on a browsing interface.

Another direct consequence of this architectural difference between both products is that Spotify provides read-only access to its song database, and in my case, that meant that curating the music library as thoroughly as I had previously done was impossible. If I spotted a typo in a track name, for instance, I would have to flag the track as having inaccurate information, in order to notify a Spotify curator to fix it. There was a shift, from owning the music and the data, to simply having access to it. In Spotify, the editorial opportunities available were limited to creating playlists.

Owning Features

Apart from Spotify and iTunes, there are countless music players of various architectural styles, and naturally, the one base trait they must all share is the ability to play music. Beyond this, each one offers a grab-bag of features to support the central task: metadata editors, smart playlists, visualizers, integrated lyric searching, etc.

I take issue with the current practices of developing software because, as I referred to in the first part, it is a labour-intensive process which requires a certain knowledge of the inner workings of computers. Not everyone is inclined to learn the discipline of computer programming, and so programmers and UX designers must step into the shoes of an end-user in order to anticipate and dictate which features are included, and which aren’t.

For example, when I find a song that is particularly appealing to me, I tend to obsessively binge-listen to it. (It can get pretty bad. I remember playing and rewinding “The Sign” on my Ace of Base cassette album at least 20 times in one sitting. But I digress.) Computers remove the need to rewind songs in order to listen to them again. Most music players will even include a handy Repeat One feature, for people like me.

Consider this: here is a feature that I know and enjoy, and also a feature which is not particularly difficult for a computer programmer to implement. To describe the logic behind the feature is dead easy: when the song finishes, play it again.

I enjoyed this feature when using Apple’s iTunes, but was dismayed to find it absent in Spotify. Luckily for me, the company introduced it to their desktop player in February of 2014, and announced it with a blog post. “You asked and we listened!” they announce.

Unfortunately for me, as of the time of this writing, this feature is still unavailable in the Android app.

I believe this perfectly illustrates another intrinsic absurdity in the process of manual software development: a feature that I use and enjoy, and which is incredibly easy to describe and implement, is not necessarily available to me. I must rely on the software maker to implement and maintain it. Apple and Spotify, for all of their talent, are limited in what they can design and implement, and so feature requests that shout the loudest get answered the fastest.

Conversely, because of Spotify’s and iTunes’ auto-updating features, there exists the possibility that the features I know and enjoy won’t be there tomorrow. Repeat One exists in Spotify on my Mac, but for how long?

Software ecosystems are in a state of constant flux. Features which were previously available or known to you might be removed, modified, or broken, with each subsequent software update. New features and paradigms are introduced all the time, as products evolve, grow, and shift their focus.

A handy but little-known feature might be removed from one version of an application to the next. Conversely, an incredibly-useful but obscure feature that would make your job ten times easier might never get built, because the software’s developer has other priorities.

These are human-motivated changes, but what about unmotivated ones, such as bugs and anti-patterns? While we’re on the subject of iTunes, have you heard about the installer for iTunes 2 which could potentially erase your whole hard drive? And not to play favourites, Spotify recently found a severe bug in their software which required every Android user to manually upgrade the Spotify app, and re-download all their previously-downloaded music. This is not a dig at either company — programming software is very tricky business. Even with the incredibly bright minds at both companies, well, shit happens.

Owning Your Computer

If you want to teach people a new way of thinking, don’t bother trying to teach them. Instead, give them a tool, the use of which will lead to new ways of thinking.

– Buckminster Fuller

I think there’s a much better way to make software, and I alluded to it in the previous article. By using computers to write computer programs for us, we can do away with many of the human errors which result in features breaking.

By delegating the rote, tedious, and error-prone job of programming to a machine (which is why we built the machine in the first place), we can focus on using our intrinsically-human gifts of creativity to design software faster, much cheaper, and without the resource bottleneck that exists today.

In much the same way as iTunes allowed me to own and curate my own music library, I’m looking to have the same experience on my computer platform at large. I can own the physical boxes that run the software, but as far as the software itself goes, I mostly have to trust that other people will do things that make my computer useful and pleasing to me. What if there were a better way?

Imagine being able to build all of your own software. In the case of a music player, you could start by having a list of hundreds of different features in front of you, each with its own checkbox. You’d go through the list, checking the boxes of features you wanted, and leaving unchecked those features you didn’t care for. At the end of the form, you’d find a button which, when pressed, would churn out a bespoke music player, tailored precisely for your tastes, having exactly the capabilities you needed.

Imagine, as well, being able to describe a feature, e.g. “the ability to listen to a single song on loop,” and have that be all of the effort required for you to have that feature built into your bespoke music player.

Going beyond that, imagine the capacity to describe a feature that no one has ever heard of or implemented before, and have it be built and crafted into your music player, e.g. “Always repeat songs by Tom Waits twice, before moving on in the playlist.”

In order to have such a software ecosystem, we need to bridge the gap between human description of features, and their technical implementation. We do this by isolating and cataloguing patterns of computer software design, and their meta-patterns of assembly. How does a feature like the Repeat-Waits break down into smaller components, (i.e. “repeat twice” and “identify Tom Waits songs”), and how do those components fit together in a meaningful way?

The translation of meaning, from human description to software implementation, is key to this new way of programming, and this is where I think the ideas of semantically linked data have a lot to offer.

The Human Side of Computing

Every day, and especially apparent from my vantage point in the Silicon Valley, more and more people are spending more and more time online. Smartphones and tablets have made computers portable like never before, to the point where connecting to the internet while riding a train is a pretty ubiquitous sight.

Undoubtedly, the human race is forging an ever-stronger attachment to computers. As technology brings itself ever-closer to the social, creative, educational and bureaucratic spheres of life (to name the most salient ones), the interface between humans and machines takes on a central role in the design of technology. User Experience (UX) designers study this very problem: how can we fashion interfaces that allow computers to become virtual extensions of ourselves; of our thinking and of our bodies?

Touch screens, for instance, have proven to be a natural and extremely intuitive way for people to manipulate on-screen objects like photos, web pages and game sprites. I constantly marvel at how very young kids seem to innately understand multitouch interfaces like those on tablets.

This is the world we live in: mothers giving their children iPads to keep them entertained and engaged during an otherwise-boring 4 hour flight.

And yet, if we look beyond computers-as-entertainment, and begin peering into the world of computers-as-tools, we begin uncovering a much different landscape.

Let’s start with what I think is a very reasonable statement: Computers are such powerful and useful tools to us because they allow us to accomplish with relative ease what might be incredibly tedious, error-prone and difficult to do without them.

To illustrate, I cite the example of William Shanks (1812-1882), who devoted many years of his life to meticulously working out the first 707 trailing decimal places of pi. It was monotonous work. He would spend his mornings generating new numbers and his afternoons verifying his morning’s work. In 1946, it was discovered that he had made an arithmetical miscalculation at the 528th position, causing the remainder of his work to be inaccurate and therefore worthless.

Today, I could calculate the first million digits of pi without even taking a single bathroom break. Or, if I really need to go, I could probably even Google it. Computers are exceptionally good at tedious, repetitive work.

The Cult of the Programmer

We therefore hold ever-higher expectations of computers. As they increasingly become extensions of our physical and social selves, we have come to expect and appreciate the newfound ease with which we can entertain ourselves, connect with others all around the world, and facilitate our daily lives.

We also expect powerful microprocessors to make astounding numbers of calculations to facilitate encryption, audio and video compression, web searching, gaming and countless other wonders of software engineering.

All along the spectrum, from the processor on one end, right out to the user on the other, we have the art and science of computer design.

UX designers tend to build closer to the human end of the spectrum, while programmers (and other highly-technical workers) build at the other end, and are much more involved with the nitty-gritty technical details of making the computer do what you ask of it.

The process of software design is all about patterns. They exist as a basic unit for writing computer code, as well as a guiding principle when designing the user interface. A pattern refers to a tried and tested way of implementing an idea. While the details of the implementation may change (e.g. the programming language in software programming or the screen size in UX design), the underlying structure behind it is based on a repeatable convention.

Patterns can exist at all scales, from the macro (e.g. the user flow through an app) to the micro (e.g. two lines of computer code), and what exactly constitutes a pattern could be the subject of debate.

Nevertheless, software design could be seen as the process of assembling a collection of software and UX patterns as building blocks, and like Lego, assembling them in an interlocked fashion to form a greater whole. A skilled programmer can then refine and modify the patterns to the degree needed to get the precise desired result. (Unlike with Lego, where taking out a piece of sandpaper and actually rounding out the edges will just get you many stern looks and chastisement from your playmates. Especially Becky.)

To my mind, we’ve already arrived at a certain contradiction: we have computers in order to save us from tedious, repetitive and error-prone work, and yet the very process of software design is at least repetitive and error-prone, and at times, tedious.

By repetitive, I mean that the same work must be done over and over again. That is, in order to accomplish a similar result, the same efforts must be repeated time and again. While there exist patterns and best practices that emerge from developing a software domain, regressions are generally commonplace.

Consider the case of web servers, which respond to HTTP requests — the basis for delivering web pages from a disk drive somewhere to your iPad screen. Web standards and best practices have been evolving constantly for over 20 years, to bring us the rich web we enjoy today.

If I wanted to build a piece of software with its own web server, and I wanted to do so from scratch, it would be a considerable task, much bigger than back in 1991, when the web was just beginning. To make it fully interoperable with today’s web, it would have to include support for cacheing, virtual hosts, streaming, reverse proxies, encryption and a whole host of other enhancements to the original web specifications. And still, in the process of building my software, it is possible that I apply the wrong code pattern, or that I simply make a typo which breaks basic web functionality. If I’m really unlucky, the bug will make its way out into the released software, and more than 20 years after the inception of the web, it is still possible to run across a web server which doesn’t implement the basic protocol correctly.

The recent Apple SSL bug is an excellent example of this very phenomenon: almost 20 years and who-knows-how-many successful implementations of SSL later, it is still possible to release a version which is completely broken in a fundamental way. Worse, the error looks like it may have been the result of a sloppy copy-paste operation on behalf of one of their programmers. Copy-pasting code is a pretty good signal that a task is rote, and in this case, that resulted in a very large and very costly software flaw.

I don’t mean to attack Apple over the flaw, but it provides a good case study into the problematic aspects of programming. It also points to the very ephemeral nature of software code, and how bugs are only fixed until they’re not.

The recent Heartbleed SSL vulnerability runs into similar themes. At its core, the vulnerability was due to a specific type of bug which has been found and fixed so many times it has been given a name: missing bounds check. (This is what could be referred to as an anti-pattern – a known problematic pattern).

Hindsight is 20/20, and it’s tempting to think that every now and then, one or two anti-patterns make their way through the software testing process and wreak a lot of havoc in the real world for a limited amount of time, and that once found, the threat is neutralized. Poul-Henning Kamp, a respected software developer and columnist for ACM Queue estimates that the computer you are reading this on may contain up to 1000 more such vulnerabilities.

It’s also possible that some, most or all of these bugs are caused by well-known anti-patterns: buffer overflows, SQL injections, OS command injections, etc. (The SANS Institute maintains a list of the top 25 most dangerous anti-patterns). The underlying issues have been so well studied that in many cases, applying the fixes is in itself a rote activity.

I am left feeling like designing software is somewhat like William Shanks’ endeavour: while we no longer do the calculations for pi by hand, we create the algorithms to calculate the digits of pi by hand, and we keep making a mistake on the 528th line.

Given the earlier contradiction in how we expect and use computers, it begs the question: Could programming be better accomplished by computers?

(Continued in part 2)

Release Notes – January 12th, 2013

This latest release of the Exxis Semantic Web Browser has been a long time in the making, and that’s because it has been entirely rewritten for speed and flexibility. On the front end, all HTML and Javascript code has been cleaned up and optimized. On the back end (i.e. under the hood), a massive rewrite of the architecture makes the core software more powerful than ever.

Now that I have reached a major milestone on the software side of the application, I will now turn my attention to the visualization of the semantic information presented, to make it much more intuitive. Lots of exciting developments are forthcoming, stay tuned!

  • Lightning-fast searches!: I managed to get search times down from over 30 seconds to less than 200 milliseconds, quite the dramatic improvement! This is fast enough for instant search! Try it! Just type a term you’d like to know more about, and without even pressing return, you’ll get a list of search results.
  • Instant Search

  • Complete architectural redesign: Pretty much everything you see has been redesigned, as well as everything under the hood. This release represents a major architectural overhaul of the core software, to support plugins. This means that it should be much easier for me and others to develop new visualizations and reasoners for semantic information. In other words, this release represents the beginning of the Exxis Semantic platform! Stay tuned!
  • Back button works!: This is surprisingly tricky to get right, in the world of web applications, but the back and forward browser buttons should work as expected.
  • No support for Internet Explorer: I’ve decided to drop support for Internet Explorer, as version 8 is very lacking in its HTML5 support. You’ll get a much better experience when viewing the site on Firefox or Chrome.

Release Notes – October 11th, 2012

This post is used to outline the changes brought to the October 11th, 2012 release of the Exxis Semantic Web Browser. Try it now, and be sure to leave feedback!

  • No more pizza!: I have done away with the basic Pizza ontology that I was using for testing, and have replaced it with the OpenCyc knowledgebase. The browser can now display information on more diverse topics such as, for example, Princeton University, the Montreal Canadiens, or, if you still prefer, pizza.
  • Search: Since the general-knowledge dataset provided by OpenCyc is much larger than the pizza dataset, it no longer made sense to list every object in the left sidebar. A search interface replaced the object listing.
  • Asynchronous AJAX: The presentation layer is now asynchronously connected to the view layer.
  • New display: I have made lots of changes to the overall user experience, with many more changes to come.
  • Separation of data and object attributes: Data attributes (e.g. strings, integers) have been moved out of the graph, into the “Properties” pane in the sidebar. The graph now only displays object properties.

Release Notes – September 21st, 2012

This post is used to outline the changes brought to the September 21st, 2012 release of the Exxis Semantic Web Browser. Try it now, and be sure to leave feedback!

  • Dynamic Navigation!: Nodes can now be clicked, to expand them and re-focus the graph. When clicking on a node, its label will first turn orange, to indicate that it is loading. Once the node has loaded, the label will turn green, and any additional nodes will be added to the graph. This gives way to a much more dynamic way to browse the graph. Try, for instance, viewing a node like DomainConcept, and clicking on nodes to explore the dataset from there.

Release Notes – September 9th, 2012

This post is used to outline the changes brought to the September 9th, 2012 release of the Exxis Semantic Web Browser. Try it now, and be sure to leave feedback!

  • Many small bugfixes: I am constantly finding and correcting software bugs here-and-there. Last week, there were 2 server outages related to odd problems with both the MySQL and RDF libraries I’m using. In this release, I have found two such bugs and implemented workarounds.
  • Node clustering: Nodes are now clustered when they share the same predicate, to simplify the visual display by reducing the number of lines.

    Before – No clusters were being drawn.

    After – A cluster of 5 elements was detected. The ‘hasTopping ~’ label is now drawn once, instead of 5 separate times.

Release Notes – September 1st, 2012

This post is used to outline the changes brought to the September 1st, 2012 release of the Exxis Semantic Web Browser. Try it now, and be sure to leave feedback!

  • MySQL backend integration: The previous version relied on an SQLite database, which was not optimal for anything bigger than a very small semantic database. This new database integration adds robustness and speed. (I will shortly add support for Postgres, as I hear it is much more optimized than MySQL — I just chose MySQL because I know it well.)
  • More consistent graph display: The focused node now appears at the centre of the canvas. Superclasses are displayed above. Subclasses and instances are displayed below. Incoming and outgoing nodes, which are not related to class hierarchy are drawn to the left and right, respectively.



  • Fewer nodes: A lot of the feedback I got was along the lines of, “I don’t get it”. I’ve therefore filtered out many of the predicates that don’t intuitively mean anything to most people (e.g. disjointWith, first, range, domain, onProperty etc.). I’ve also filtered out most of the Restriction objects from the graph. These connections will re-appear in later releases, presented in more functional ways.

Page 1 of 2

Powered by WordPress & Theme by Anders Norén