May 13, 2010

Stakeholder alignment

I’m designing the UI for an application, and people here have different ideas about what this product should do. How am I supposed to design when I don’t know what I’m designing?

This is a common problem, especially when requirements haven’t been written down and agreed on by the project and business stakeholders. We recommend doing a stakeholder survey early in the project, to help align the stakeholders. The survey should ask each stakeholder to define:

• key business goals of the project (increase user adoption, reduce support costs, etc.)
• user experience metrics (efficiency, learnability, etc.)
• which personas or user types to target
• key use cases or user tasks
• the top features and functions
• relevant technologies that will affect the user interface
• branding guidelines
• etc.

Try to limit the number of stakeholders here to 6 people. Good luck with that.

Some of the stakeholders won’t have data or opinions about some of the questions, and that’s fine. They can skip those questions.

Analyze and summarize the data, and present the results to the stakeholder team. Get everybody in one meeting, and point out where there is agreement and disagreement. Where there’s disagreement (and there might be a lot), get the stakeholders to discuss, iron out the differences, and achieve alignment/Nirvana.

Often, stakeholders disagree because they rely too much on their personal opinions and guesses instead of on data from customers and end-users. You can make this whole process go smoother if you are armed with good data to inform the team’s decisions.

Filed under: Design, Process_and_Strategy | Permalink

January 11, 2010

Finding study participants

How can we find people to be in our studies?

Study participants can be found in many ways.

Start by defining the types of users you want for your study. A typical persona will not be enough here. For example, if you have a persona called “Amy”, a 27-year-old office manager who likes romantic comedies, etc., whom will you recruit for your study? People named Amy? 27-year-olds? Office managers? Romantic comedy fans?

Instead, for each user type (perhaps “Office Manager” in this case), you need to define the relevant set of demographic and psychographic criteria that represent the actual users you are targeting. This might include:

• an age range
• a minimum level of education
• a list of possible job titles (or, better yet, key job responsibilities)
• the gender breakdown (% female and % male)
• a minimum level of experience in the job
• a geography (for example, “U.S. only” or “evenly split between North America and Europe”)
• an income range
• specific attitudes, beliefs, motivations and interests that matter
• etc.

Next, determine how many participants from each user type should be included in your study. The number will be based on: whether you’re doing a survey, informal usability test, etc.; your budget; how confident you want to be that the study data are “correct”; and other factors.

Be sure to recruit in the proportions that match your target users—if 90% of your website’s targeted users are women, then 90% of your participants should be women. Also, the best data will come from people who are selected randomly from the population of people matching your criteria. So go beyond family, friends and co-workers.

The best ways to find participants will vary depending on the people you’re targeting, the type of study, and other factors. Sources for potential participants include:

• public places (the mall, the street, etc.)
• market research firms that recruit like this for a living
• referrals from friends, family and previous study participants
• trade shows and conferences (good for domain-expert users)
• your customer database
• newspaper or online ads you place
• religious or not-for-profit groups (who may help you recruit in exchange for a donation)
• social networking sites
• that User Group you so wisely created earlier for exactly this purpose.

When recruiting, have potential participants answer a questionnaire to see whether they are in fact good matches for your study.

Include surrogate users (people who know or claim to know the real users) only as a last resort, and interpret their data carefully. They might lead you down the wrong path.

Filed under: Usability, Process_and_Strategy | Permalink

October 28, 2009

Competitive Usability Testing

We’re redesigning our site, but there’s internal debate about whether the new design is any better than the current site. What’s the best way to determine which one is better?

Rather than continue to debate internally, you should get data from your end-users so you can make an informed decision. The guy down the hall has an opinion, and so do you, but your users’ responses matter a lot more.

Many techniques can help you figure out which design is better, but we’ll discuss two here. Both are types of “competitive benchmarking”, also known as “competitive usability testing” or “A/B testing”.

First, you could run a competitive test based on the think-aloud methodology. In this test, a user performs tasks with Version A of the design (or site, etc.) while thinking out loud. You make note of what the user liked and didn’t like, why they failed to complete certain tasks, where they got lost or confused, etc. Then the user does the same thing with Version B of the design while you observe. At the end, you ask the user which version they preferred and why. Do this with other representative users and look for patterns in the responses. The winning design usually emerges (“8 out of 10 users preferred Version A, and Version A was easier to use”), along with plenty of ideas on how to improve it.

Second, if the new design is already implemented, you could run a live test in which some percentage of visitors to your site get the new design while the rest see the usual design. You look at web analytics data and metrics you care about (for example, conversion) to decide whether the new design is “better”.

We recommend using both of these techniques together (the first during the design prototype phase and the second after implementation). But either technique is better than none.

Tips for competitive testing include: using the same tasks and the same metrics for both versions; asking users “why?” when possible; and alternating the presentation order across test participants if they see both versions (half will see Version A first, and the other half will see Version B first). It’s also good to have users compare designs at the same level of fidelity and interactivity—don’t test static wireframes against a live site and expect to get valid results. In this case, you could create static wireframe pages from your live site for the test.

Expero offers a training course on Competitive Usability Testing for people who want to learn the techniques in depth.

Filed under: Usability, Process_and_Strategy | Permalink

October 16, 2007

User data sources

We get feedback from users when we’re able to do usability tests, which is not often enough. What other sources are good for user data?

Companies often fail to take advantage of internal sources of user data. In defining requirements for the next release, do you talk to Tech Support or Customer Service to identify the most frequent or critical usability issues they’ve heard about from users? Those teams also are on the front line for hearing about new features that users want.

Technical Writers and Documentation Specialists also can be gold mines of information. If those folks are not already on your team, find them and ask their thoughts. Chances are, the things they have a hard time documenting are good issues to be explored.

Marketing and Product Management might have valuable data for you from surveys, focus groups, interviews, etc.

If your organization has a Training group, ask them what’s working and what the users want changed. Observe the training sessions yourself, too, and listen to people’s questions.

How about Sales Reps and Consultants who go out in the field? Try sending a User Experience Specialist on customer visits with them, to emphasize how important the customers are to your company, and to solicit feedback and requirements in a way that’s probably different from how the Sales Reps and Consultants do it.

However you do it, don’t rely on any single source to provide you with user data. Get feedback from multiple channels, compare and contrast the data across channels, and look for patterns and ”a-ha” nuggets in the data.

Filed under: Usability, Process_and_Strategy | Permalink

August 21, 2007

Remote Usability Testing

What are the pros and cons of remote usability testing?

Remote usability testing is an excellent method for getting feedback when study participants, facilitators and/or observers are in different physical locations. Using web conferencing software such as WebEx, GoToMeeting or UserVue, remote testing enables screen sharing among everyone involved in the study. When the study participant has control of the screen (in presenter mode), everyone can see in real time what that user is doing. A simultaneous telephone conference call enables people to hear what the user says as he or she works through tasks.

This method offers convenience and can save a great deal of time and money, as you can run sessions with users anywhere in the world without incurring travel costs. It’s also useful for enabling distributed development teams to observe sessions (we’re big fans of multi-disciplinary team members observing sessions live, whenever possible).

A few downsides to remote testing should be considered. Unlike testing in person, remote testing does not let you see the expressions on a user’s face, unless the user has a webcam.  Facial expressions, gestures and other body language can be important sources of data. In addition, remote tests require slightly more preparation than in-person tests. For example, you need to ask potential participants about firewalls, operating systems and other system requirements to make sure they’ll be able to access the web conferencing software. Also, the web conferencing software adds another layer of technical complexity to the session, which may frustrate or confuse less-savvy Web users.

For the most part, though, the cons of remote testing are minor when you consider the benefits.

Filed under: Usability, Process_and_Strategy | Permalink