Usability Study – Is 5 Enough?

When design is agile & user centered and video is your friend, you can find your biggest flaws quickly with even less than 5 users. 

As a start-up girl I know that often usability studies have to be conducted as the sole facilitator, especially when design decisions have to be tested and time is never enough. I have often pushed my limits on legibility and speed writing when conducting a usability study and I still do. Also I’ve learnt to rely on instinct and impressions, letting my subconscious to do the analytics while I gather as much information as I can. But lately I tried mobile video recording as the extra eye on the job. There are many advantages of having a video recording of your usability test.

The first one is that you get two pairs of eyes reading your data from two angles. You from across the desk reading facial expressions and eye movement and the video from over the shoulder recording screen interactions and the user’s ‘think aloud’ thoughts. The second is that you get a tangible record of actual screen interactions to add to your inferences which you can keep going back to and watching again and again. And you get a tool to sell your inferences forward to stakeholders. The 3rd one may not sound like much but put it together with the prioritisation engine that is project management, and you probably have the most powerful tool in your hands. Lastly, it gives you validation with even a really small set of 3 or 5 users.

But is 3 or 5 enough? The answer that I have realised is – Yes. Absolutely. When testing mid development on a half baked product, even 3 is more than enough. With video and two pairs of eyes, you often get repeat impressions that may manifest in emotions and reactions, or screen hesitation and interactions or words. The biggest problems are faced by the largest number of users, its simple mathematics. The more clues you get to form impressions with, the more validation or invalidation of hypotheses you can arrive at. So in essence, the addition of a casual video observer strengthens and speeds up usability study to a very large extent.

But really is 5 enough? When you cover your target audience personas well in a broad spectrum group, you can arrive at decisions on things like screen affordance, understanding, expectations from the product lightening fast. After all, if a button looks like a banner ad, it will look the same to most people. If finding the next step is difficult, it will be more or less difficult for every one. If the screen has contrast issues, most people will have legibility problems with it. The biggest usability flaws come from the consistency of human computer interactions. And finding the 20 biggest usability flaws fixes 80% of the usability issues in an application (The Pareto Principle of UX).

In fact, in my experience and that is consistently felt across the globe also, more than 7 users is a waste of time and energy. Sure, you may find more issues with more people, but the consistency of the issues decreases with more users. And the biggest flaws have the most repeats. So in essence with more than 7 users, you are repeating the same thing again and again and impressions are getting more and more diffuse and difficult to form.

So if I had any advice for my next usability test, or yours, I’d say – crank up the mobile camera, wipe your SD card clean, decrease the quality of video, and shoot away. Video is a friend. And yes, 5 users are enough!

Data Driven Design – Part II – Personas as Data Collection Tools

Personas are people. Data comes from people. Whatever data it may be. You may look at data about readership, usage, drop-out rate, heat maps, contextual studies or search terms analysis. Its when you start to visualize usage patterns as belonging to personalities that you can create & use personas.

Personas help you connect with your target audience. We as human beings intrinsically know people. We know that little children need to be protected and encouraged, we know that teachers have ambitions for their students, we know mothers want their children to be healthy and achievers. So when we find who it is that will be or is using our product, we know, can take an informed guess at or ask them what they want. For that you have to identify with personas as people. The more you identify with them, the more you think about them, watch them work or read tons of data that they generate, the more you come to know what is it that they want from your product and from your brand. Lets take a look at the three main type of personas that you may build for a brand, a product or a feature.

Persona for Budget Designs1) Target Audience Persona

A Target Audience Persona is made before you conceptualise & design a product. You research & visualise the target audience, recruit them into usability studies, talk to them and observe them to see what is it that will make the product something that they cannot do without. I’d call this way of working Darwinian because when you create a persona and a value proposition that may appeal to them, it gets refined over time. You are back at the drawing board after every couple of weeks trying to evolve your persona to match the product usage and evolve the product so it matches the persona even more. It works though.

At a mobile youth magazine, personas were used to set the tonality and content of the SMSs & MMSs they would receive. Gleaned from a contextual study of the digital habits of the generation, the personas helped set up the content calendar as well as engagement strategy. Since the magazine was digital, personalized and mobile, the content strategy was refined with real time data and results, as were the personas. As the personas evolved over time, as did the content strategy. A successful approach as illustrated by the exponentially high response rates. The average response rate calculated over hundreds and thousands of users, was between 30 & 80%. Here’s a sample of how the content strategy was arrived at:

yourstory-vserv-report2) User Persona
The second route is to create personas based on traffic and behavioural analysis. This is the most travelled road, but that does not make it any less useful. A study released by Vserve (left) about the personas of Indian app users is so accurate that you can actually guess the value of people’s phones using the persona as a reference. I’ve tried it in a Persona Workshop. Works like a gamble but the probability of you finding an accurate match is so high, you hit bullseye one in 4-5 times. Why don’t you try it? Check it out anyway…the personas are quite interesting.

Working with a persona journey or user walkthrough yields insightful usability issues and solutions. A journey for Indian Gifts Portal diagnosed issues in gift findability and a solution was designed using PET guidelines on the basis of personas identified. A gift suggestor, prototype tested in live conditions yields sky high conversion rates for them. Personas gleaned from traffic & audience analysis are a useful tool to keep handy for immediate design decisions like cart optimisation, mobile responsive design, content marketing, social media strategy etc.

2) Proto-Persona

The basis for this kind of persona is human experience and interactions. The data gathering for a Proto-Persona is done in workshop format. Stakeholders & customer facing employees of the product sit together to talk about their customers and their experiences with them. The evidence gathered in this format, although anecdotal, also translates to rich insights about what the information and content needs of their users are. It is good practice to validate as many data points as possible for each persona through segment analysis later. I call them echoes or shadows of the personas. You see their data footprints all around you and you know you’ve done it right. Multimedia consumption patterns, social sharing patterns, data tells you the story of that segment of users.

Proto-persona echoes can be heard in more engaged customers as well. It is most gratifying to see them in the customers who have bought, subscribed or donated. A recent persona based redesign for Down to Earth, an environmental magazine, brings in more profiled customers as well as more engaged readers who match their personas. Bringing home the validity of the personalities crafted in a proto-persona workshop.

No matter how or for what purpose a persona has been created, its always a good idea to hold on to them. Go back to your personas and their stories every time you refine the product. Get to know them better. Hang them on your walls. I like to keep track of mine in editable power-point format. Here’s a template, if you want to use a pre-made one, else its quite easy to make in powerpoint using shapes.

Next up in Data Drive Design -III is a the story of recommendations and personalisation. Exceptionally suited for mobile application design. Next week, if I can make it. Adios.

Data Driven Design – What and How to.

Data & Design. Left brain or right? Confusing isn’t it? It shouldn’t be. Since the beginning of the digital revolution data has driven design decisions for every digital designer out there. Have you ever redesigned a screen because the drop out rate was too high? Well that was a data driven design decision. Yawn…we’re already doing it? So whats the big deal about Data Driven Design? The difference between doing that in 2000s, and using data to drive UX design today is that measurement metrics that go beyond just a single metric. Today data comes from several sources – traffic logs, search term analysis, sentiment analysis and more. A UX designer uses it to cull out insights that lead to better experience for users on your site. So you are not guessing at the problems in cognition or flow but making informed decisions using data from multiple sources on how to solve them.

There are various ways of using data to drive UX on the site or app. Both qualitative data like the output of a contextual study or quantitative data like audience behaviour analysis helps you in choreographing a product that is exactly designed to fit a an existing gap in the market. Or it can diagnose a gap in an existing product and give you the tools to fix it.

Here’s a low down on quantitative data that can give you clues to what’s going on your site. The first and the foremost is traffic logs. That goldmine of info is like an open book, if you just know how to read it and analytics tools make it easier and easier to read by the day. Google Analytics also has tutorials on how to interpret the streams of data that you see there.

The second is search term analysis – its possible to measure the demand for information or utility by analysing search terms surrounding your key propositions. For Indian Gifts Portal, huge amounts of search terms were analysed in terms of incoming hits and number of searches on google to arrive at a design for the mega menus which mirrored the users intention map.

Traffic & Search logs together can give you a peek into what your user is looking for. For example while designing the web site for a major hospital brand, we analysed search & traffic trends to find what the users were looking for. Location specific information. Doctors in a location. Hospitals in a location. Medical specialities in a location. Users are looking for a location specific answer to their medical use case if they can find it. They were looking for hospitals and their locations especially on mobile as shown by the hits to ‘Contact Us’. The solution mirrored the problem, in the new design, maps and location finding are a large part of the solution. The website uses IP redirection to show the user what is most important to him upfront – hospitals, doctors and specialities in his area. In a non-redirected phase, the user needs 5-6 clicks to reach a doctor in his area, using the IP oriented reach mechanisms he can get to it in 2-3 clicks or less.

Similarly error logs can give you deep insights on the problems a user may be having while filling up a form. Lets face it – forms are essential characteristics of conversion, be it simple share buttons or complicated purchase forms. A study of error logs will give you clues on how guideline text can help you in converting more online customers. Giving a sample or giving clues to what should go into a text field can make the difference between the submit and the cancel button. For Jobsahead, a jobs site that I worked on, tweaking help text of an open text field led to an increase in the influx of resumes, more resumes meant more business. 

Then comes sentiment analysis – although this is more of a social and content strategy data point but a lot of times sentiment analysis will give you a bird’s eye view of how an application is being accepted and used across the board of audience types.

In addition to quantitative data – UX Analysts use qualitative data culled from usability studies, contextual research, user interviews and walkthroughs to arrive at design decisions every day. The biggest of all UX activities led by and using data is Personas. The next post, we will talk about how personas use data to validate user actions and intentions. Personas are by far the best use of qualitative data in multiple dimensions. Next week, find out how you can use data both qualitative and quantitative to arrive at Personas that not only represent and allow you to identify with your target audience but also become the guiding light to make the right design decisions.