By Michael Heraghty on September 13th, 2013
Many of my clients have never previously carried out usability testing. Even when I tell them about how successful testing has been for my other clients, I can sense that they are nervous. For many people, usability testing feels like a step into the unknown.
To overcome the client’s fears, I use a clear testing method, which I talk through with them at the outset. The method involves a number of steps:
I recruit a minimum of five and a maximum of seven users. Clients who are more familiar with focus groups or surveys sometimes think this number is too low. I explain how, for usability testing, a small test group captures most of the big usability issues. After that, it’s the law of diminishing returns. This principle is well documented.
I try to conduct the testing in a single day. I allow 45 minutes per test, and schedule each user to arrive at one-hour intervals. This gives me some time for set-up and breaks.
On occasion, one or two participants won’t make it. Seven scheduled participants is ideal, because if I have two no-shows, at least I still get five tests.
It can be expensive to recruit “the right” people, especially if you have to hire a marketing agency. Recruiting from friends and family – yours or the client’s – is usually quicker and cheaper.
Many in your network may fit the desired demographic profile. While that’s not ideal, it’s not the end of the world either. Aim for a profile match, but take what you can get.
Sometimes, the client will suggest their own “unbiased” staff members – for example the company’s accountant. Some usability testers reject all staff members, on principle, because they are too close to the proposed design. However, if the staff members aren’t familiar with the interface or concept being tested, you may be foolish to reject this offer. Staff members often make good participants, and testing with more neutral staff members promotes interest and buy-in to usability testing generally.
Ask the client to list the main tasks they want the user to perform when visiting their website, or using their software.
This gets the client involved from the start, and gets them thinking about the user’s goals, something they may never have done before. It also helps to demystify the testing process and makes them realise it is straightforward.
If the client suggests tasks such as ‘book a flight’, tell them to break this down into more specific tasks, like ‘Research flights from Dublin, Ireland to Las Vegas, leaving on 22 June’ and ‘Find the flight with the shortest flight time.’
Together, you should agree a list of no more than 10 tasks.
Most clients have a meeting room or office they can make available for a day. They often have a receptionist or another cheerful member of staff who can act as the meet-and-greet person for participants as they arrive.
Testing on the client’s site
– eliminates the cost of hiring a testing lab
– eliminates the cost of a meet-and-greet resource
– increases the likelihood that the client will observe the tests
– helps to demystify the testing process
If the client agrees to observe the testing, you won’t have to convince them later about the veracity of your usability report – instead, you can simply discuss your joint observations.
I like to have a single observer present in the testing room. Their role is to observe and listen, not to interact with the participant – although you may allow them to ask questions during the post-test interview.
If the client wants to have multiple observers, ask them to provide a separate room, preferably a meeting room with a projector. Using screen sharing software, the observers can watch the proceedings in this “observation room”. I like to relay the audio from the test room using the company’s speakerphone system, i.e. by leaving the phones in both rooms off the hook. That way, clients can hear what’s going on via the phones, and see what’s going on via the screen-sharing software.
The usability intimidation barrier is real. One way to help overcome it is to eschew expensive testing software and equipment, such as eye-tracking labs and two-way mirrors,and instead use simple tools and methods that won’t sound alien to the client.
My preferred usability testing tools are
– laptop, displaying the interface to be tested
– notepad for live note-taking
– screen recording software, optional
– screen sharing software, optional – for observation room
– speakerphone, optional – for observation room
I find it difficult to take detailed notes if I’m moderating the test, but I scribble enough notes to capture the big issues. If the client is observing, I ask them to take notes too.
I use screen recording software if the client wants detailed feedback. Recording allows me to look deeper into some of the issues. I don’t play back through all of the recordings – I only go to where my notes prompt me to look deeper, or where I feel my notes are missing details.
Some participants may not be available to travel to the test venue. Tools such as WebEx or Skype allow remote screen sharing, and do not require any additional functions on the client’s side – they just need a monitor and a phone.
Unless you use a webcam, you will not be able to see the participant’s facial expressions. Nevertheless, I find remote testing provides good results and uncovers many of the major issues.
The quicker you can show results to the client, the more impressed they will be by the efficacy of usability testing. In addition, the participants’ behaviour will be fresher in your mind.
After observing only a few users, both you and your client will have a strong understanding of the major usability issues. Hold a debrief meeting with your client as early as possible after the testing to compare notes.
If your client has not observed the testing, get your formal report turned around quickly. The aim of the report is to communicate and interpret what you observed.
If you are creating a formal report, make it visual – use screenshots and short, concise annotations.
I like to create reports using PowerPoint. That way, I can deliver the report in a workshop style meeting, where the client and I can discuss the findings.
Also, love it or hate it, PowerPoint is still used extensively in the business community. As a deliverable, it’s acceptable.
For each major usability issue I present, I offer a solution.
I use Photoshop to hack a rough screenshot of the proposed solution. Alternatively, I create a wireframe of the solution using Balsamiq Mockups.
Sometimes I will do a quick competitor analysis, and get suggestions of alternative solutions from other websites.
I discuss my proposed solution with the client and get their feedback. The client does not always agree with all of my proposed solutions – but the solutions get them thinking of their own alternatives. Usually one of the client stakeholders at the workshop will know of another website that has a neat solution that I won’t be aware of. The client knows their own industry the best. But they need guidance from a UX practitioner – in the workshop, I always try and explain to the client what works and what doesn’t work, as far as I know.
The key words here are “as far as I know”. If the client has a strong idea for a solution that I don’t think is a great design concept, I offer my advice, but suggest that we should mock it up and test it, if they are passionate about trying it out. And so the next round of usability testing begins J
It is important to have a clear plan about how you are going to conduct your usability testing, but you should also remain flexible and open to ideas – be willing to see things from the client’s perspective.
Most user experience workers are good at putting themselves in the user’s shoes. But to be a user experience professional, you must put first yourself in the client’s shoes.