Topic: Usability Test Plan - Layer2

Message

Layer 2: Further Background

If you've got a little more time to put into planning, there's plenty more detail you can add to provide a clearer picture of what the testing is for and what you'll do during the test.
User Profiles and Screener

Although the logistics element from layer 1 should reference the kinds of people you'd like to recruit, you can always provide more detail. The more complex the test, the more detail you'll want to give because you may be testing different user groups. Defining profiles for each group allows you to effectively design an appropriate test for each group. If you've done user personas (described in the previous chapter) you can draw straight from those. Here's a user profile for a parenting web site:

The Day Job Parent needs to work outside the home. He sees his kids in the morning, maybe drops them off at the bus stop, and then sees them again in the evening, when he comes home from work. The Day Job Parent may steal a glance at this web site during a lunch break, seeking advice on specific child-rearing problems. He may spend a bit more time on the site in the evening after the kids are in bed. In the Day Job Parent's house, both spouses work. They are both familiar with Internet technologies, and have shopped and researched health issues online.

High-level descriptions of users are one thing, but actually finding people who meet those criteria is another. If your test is formal—that is, you're recruiting a large number of people outside your immediate user group—you need to put together a screener. A screener is a set of questions to ask potential participants to see if they fit your needs for the usability test.

Some screener questions are pretty straightforward, asking about experience with using the Internet or shopping online. Some will be more specific to your particular application. For example, for the fictitious online health encyclopedia, your screener might seek out people who have recently been to the doctor, or who visited a health site in the last three months. Such questions will help narrow the pool of applicants to those who best match your target user group.

Usually, recruiting from the general public is done by a recruiting agency. They have a large list of people to draw from. If you're using an outside agency, you'll need to provide as much detail in the screener as possible. Indicate when a particular answer to a question should eliminate a potential participant. Indicate quotas for particular answers. You may want, for example, half your participants to have visited a health site in the last three months, and half to never have visited a health site. Recruiting methodologies aside, you need to communicate the makeup of your ideal group of participants to the recruiter. A screener is the best way to do this.
Figure 3.3

Figure 3.3 In this excerpt from a screener for a web site on parenting, the usability test needs advanced Internet users only. The group of participants will vary by the age of their children, and whether both parents work or not.
Pre-Test and Post-Test Questions

Usability testing best practices suggest that you ask users questions before and after the actual test. At the beginning of the test, before showing participants any screens or prototypes, you can establish their expectations with respect to the web site and their general level of experience. You can clarify their motivations for using your site and flesh out your profile of them.

The format for these questions will vary depending on your methodology. Some usability testers ask the questions of the user in person, recording their answers by hand. Others give participants a questionnaire that they can answer separately. (This approach allows the facilitator to spend less time with each participant.) If you decide to let participants answer the questions on their own, you may want to separate these questions into a different document because it will be something you hand to them. In that case, you may need to format it differently, and—if the participant is to answer the questions without intervention from the facilitator—you'll certainly want to be explicit about directions.

Pre-test questions can help you get a handle on the user's expectations and experience before seeing the site for the first time. Questions after the test can gauge overall impressions and allow general pain points and suggestions to surface. Formatting for pre-test and post-test questions can include open-ended questions and questions with predefined responses.
Examples of open-ended questions:

We're developing a new system to support your sales process. What are the most challenging tasks in the sales process for you?

The web site we're building is called FluffyPuppies.com. What do you think you'd find on this web site?

What system do you currently use to manage your family videos?
Examples of questions with predefined responses:

On a scale of 1 to 7, where 1 is strongly disagree and 7 is strongly agree, how would you rate this statement: I found the web site easy to use.

What level of Internet user are you? Beginner, Intermediate,

Advanced

How long have you been involved with the sales process?

Regardless of how you deliver the questions to the participants, you should write the questions exactly as you intend to ask them. This can helpg ensure that you ask the question the same way every time.
The Script

Layer 1 calls for defining the scenarios for the usability test, though it leaves the format pretty open. For the second layer, you can add more detail, depending on the complexity of the test. You can look at the script as a hierarchy of information. At the highest level are the scenarios, which re-create situations in which participants might actually use the product. Each scenario includes a series of tasks, and each task describes what will be read to the participant and the background material for the facilitator. The task or scenario may also include specific follow-up questions and notes to the people developing the prototype.

Author	Message
priya Newbie Joined: 17Feb2007 Online Status: Offline Posts: 1	Topic: Usability Test Plan - Layer2 Posted: 19Feb2007 at 2:05pm
	Layer 2: Further Background If you've got a little more time to put into planning, there's plenty more detail you can add to provide a clearer picture of what the testing is for and what you'll do during the test. User Profiles and Screener Although the logistics element from layer 1 should reference the kinds of people you'd like to recruit, you can always provide more detail. The more complex the test, the more detail you'll want to give because you may be testing different user groups. Defining profiles for each group allows you to effectively design an appropriate test for each group. If you've done user personas (described in the previous chapter) you can draw straight from those. Here's a user profile for a parenting web site: The Day Job Parent needs to work outside the home. He sees his kids in the morning, maybe drops them off at the bus stop, and then sees them again in the evening, when he comes home from work. The Day Job Parent may steal a glance at this web site during a lunch break, seeking advice on specific child-rearing problems. He may spend a bit more time on the site in the evening after the kids are in bed. In the Day Job Parent's house, both spouses work. They are both familiar with Internet technologies, and have shopped and researched health issues online. High-level descriptions of users are one thing, but actually finding people who meet those criteria is another. If your test is formal—that is, you're recruiting a large number of people outside your immediate user group—you need to put together a screener. A screener is a set of questions to ask potential participants to see if they fit your needs for the usability test. Some screener questions are pretty straightforward, asking about experience with using the Internet or shopping online. Some will be more specific to your particular application. For example, for the fictitious online health encyclopedia, your screener might seek out people who have recently been to the doctor, or who visited a health site in the last three months. Such questions will help narrow the pool of applicants to those who best match your target user group. Usually, recruiting from the general public is done by a recruiting agency. They have a large list of people to draw from. If you're using an outside agency, you'll need to provide as much detail in the screener as possible. Indicate when a particular answer to a question should eliminate a potential participant. Indicate quotas for particular answers. You may want, for example, half your participants to have visited a health site in the last three months, and half to never have visited a health site. Recruiting methodologies aside, you need to communicate the makeup of your ideal group of participants to the recruiter. A screener is the best way to do this. Figure 3.3 Figure 3.3 In this excerpt from a screener for a web site on parenting, the usability test needs advanced Internet users only. The group of participants will vary by the age of their children, and whether both parents work or not. Pre-Test and Post-Test Questions Usability testing best practices suggest that you ask users questions before and after the actual test. At the beginning of the test, before showing participants any screens or prototypes, you can establish their expectations with respect to the web site and their general level of experience. You can clarify their motivations for using your site and flesh out your profile of them. The format for these questions will vary depending on your methodology. Some usability testers ask the questions of the user in person, recording their answers by hand. Others give participants a questionnaire that they can answer separately. (This approach allows the facilitator to spend less time with each participant.) If you decide to let participants answer the questions on their own, you may want to separate these questions into a different document because it will be something you hand to them. In that case, you may need to format it differently, and—if the participant is to answer the questions without intervention from the facilitator—you'll certainly want to be explicit about directions. Pre-test questions can help you get a handle on the user's expectations and experience before seeing the site for the first time. Questions after the test can gauge overall impressions and allow general pain points and suggestions to surface. Formatting for pre-test and post-test questions can include open-ended questions and questions with predefined responses. Examples of open-ended questions: We're developing a new system to support your sales process. What are the most challenging tasks in the sales process for you? The web site we're building is called FluffyPuppies.com. What do you think you'd find on this web site? What system do you currently use to manage your family videos? Examples of questions with predefined responses: On a scale of 1 to 7, where 1 is strongly disagree and 7 is strongly agree, how would you rate this statement: I found the web site easy to use. What level of Internet user are you? Beginner, Intermediate, Advanced How long have you been involved with the sales process? Regardless of how you deliver the questions to the participants, you should write the questions exactly as you intend to ask them. This can helpg ensure that you ask the question the same way every time. The Script Layer 1 calls for defining the scenarios for the usability test, though it leaves the format pretty open. For the second layer, you can add more detail, depending on the complexity of the test. You can look at the script as a hierarchy of information. At the highest level are the scenarios, which re-create situations in which participants might actually use the product. Each scenario includes a series of tasks, and each task describes what will be read to the participant and the background material for the facilitator. The task or scenario may also include specific follow-up questions and notes to the people developing the prototype. Post Resume: Click here to Upload your Resume & Apply for Jobs

	IP Logged


One Stop Testing Forum : Software Testing @ OneStopTesting : Test Plans @ OneStopTesting