The Zen of Usability Testing

My Boomworks project team recently went interstate for some usability testing. Session after session of introducing users to our design and seeing how they interpreted everything certainly left me feeling exhausted, and I was only taking notes. We saw some users blitz through parts of the site which we were still working on, and saw others stumble in places that we thought we had sorted.

At a restaurant after the second day of testing, we sleepily awaited our food and started talking about the day that was. It was no doubt the unique blend of fatigue, synapses firing overtime, and choice grapes from the Ballarat region that got us to talking about the Zen of user testing.

We postulated that Zen could be achieved by testing with an open mind and putting one’s own ego and preferences aside to listen to and facilitate the user’s needs. While the conversation begun with a facetious tone, it quickly progressed to a higher plane of seriousness. There are always parts of a design that we creators think are particularly clever – but if the people the site was designed for don’t ‘get’ them, then they need to be dropped, or at least tweaked. The other way of looking at this is that empathy needs to overrule any solution that is more convenient or impressive.

If we only designed sites for people in our industry, the results would be quite predictable. On the other hand, trying to design things to work in a way which is logical to someone who doesn’t know the limitations of our software is an enormous challenge. This is when we are pushed to make really clever stuff happen.

The almost meditative channelling of testing Zen typically lasts for days at a time, in 60 minute blocks; with the net result being a thoughtful reassessment of one’s offering to the end user. It’s a focused and selfless pursuit of understanding the needs and desires of candidates representing the millions of end users.

After each round of testing, the next step is to take a look at how user feedback impacts the existing designs. This can be a massive task, which can be all the more difficult if you’ve forgotten to leave your ego at the door. The role of testing is to see how well the design works, and what can be improved. It’s vital to put everything you have into a design – but to selectively ignore inconvenient findings is to forbid the design from growing.

The most difficult part of this process is to compile all of the (seemingly contradictory) feedback into a single direction. This becomes easier by listening to the results themselves. Sure, it’s great if everything goes as expected; but if they don’t, the next design solution needs to acknowledge the results – convenient or not.

It’s always interesting to observe the observer; not everyone who undertakes usability testing sessions seems to be able to achieve a balance of selflessness and personal investment. It’s challenging, exhausting and confronting, but I am looking forward to my next design validation sessions, and striving for the revelations of testing Zen.

6 thoughts on “The Zen of Usability Testing

  1. An interesting read! Two comments:

    I watched an interview with M Night Shyamalan where he said “Always be prepared to leave your best scene on the cutting room floor”. His point that it may be the best shot, best directed, best acted… but if it does not advance the story then it should not be there. This is similar to designers and UX specialists hanging on to design features because they are cool or clever – even though they may not be relevant or helpful to users.

    Also, its a great idea to use independent people for user testing where possible, so there is no chance of “bias”. I say that as someone who most of my clients user-testing myself, but it is always a good option if there is time and budget :)

  2. Pingback: Some links for light reading (15/8/12) | Max Design

  3. Thanks for sharing your thoughts :) The imagery of a perfect shot sitting on the cutting room floor really hits the point home. It goes against one’s first instincts to not put forth the work that will make them look more impressive – but sometimes that’s what will benefit the situation most.

    It’s always interesting to get people in testing who aren’t from our industry. They’re rarely interested in how easy or hard a piece of functionality is to make; they are interested in which features will make their experience more fun or easy.

  4. Pingback: Design Focus: Deconstructed Sliders | Devlounge

  5. As Russ indicated if it doesn’t help the user etc – it could be brilliant design/coding etc – but it’s not worth including (irony there also given M Night Shyamalan has made some of the worst films in history – doesn’t follow his own advice, or maybe there weren’t any brilliant scenes to leave on the cutting room floor lol).

  6. Pingback: Weekly Roundup of Web Design and Development Resources: August 19, 2012

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">