Topic: Creating Test Plans - Layer 1

Message

Creating Test Plans

        If you had to conduct a test tomorrow, there are three basic elements you'd need (beyond a group of users, a place to conduct the test, and—oh, yeah—a set of designs to test). The simplest test plan—described in the elements of layer 1—consists of a set of goals, a description of the logistics and method, and a set of questions to ask the participant. As stated, these are pretty broad categories, and therefore suitable for informal testing. The more formal the testing, the more details you'll need. Fleshing out the test script is described in layer 2. Further details around the logistics, described in layer 3, seem somewhat extraneous but your stakeholders may demand them.
Layer 1: When You Need to Test Tomorrow

The basic elements of a usability test answer three questions: What do we want out of this test, how will we conduct the test, and what will we ask users during the test? With those questions answered, you've got the barest information necessary to make a test happen.
Test Objectives

In planning a test it's easy to let things get out of hand. Without any direction, you'll find yourself asking users questions of increasing irrelevance. Explicitly stating a direction in the test plan, as in other documents, allows you to keep the design of the test on track. The script should be derived from the objectives and everything you ask the user should take you closer to meeting your objectives.

Unfortunately, people designing usability tests frequently gloss over this part because (1) it's hard, and (2) they want to dig into the script. Consider this: Your script is meaningless unless it is working toward a specific purpose.

Because the test objectives drive the discussion between the test facilitator and participant, you can make them pretty specific. Too often, the test objective looks something like this:

To determine whether our web site is easy to use.

You'll forgive me if I refer you to the Wikipedia entry on "duh." You can use a simple strategy to make your objectives more specific—think about why people are coming to your site. (If you created the personas described in the previous chapter, this should be easy.) By basing test objectives on user motivations, you get objectives that look more like these examples for a fictitious health encyclopedia:

To determine whether people who come to the site with a specific illness in mind are satisfied with how they located relevant information.

To determine whether people who are doing research on behalf of another person can easily refer someone else to the infor mation they find.

To understand how people who have just been diagnosed with an illness want to revisit their research.

These objectives may be used together—nothing says your test must have only one objective. Once you've written up a couple, you can do a quick litmus test. If you state the objective in the form of a question, is it an answerable question? To take the first (bad) example, the test designers could ask themselves "Is this site easy to use?" which would not be an easy question to answer. On the other hand, you could probably easily answer the question "Are people who have a specific illness in mind satisfied with how they found relevant information?"
Test Logistics

Your plan needs to describe the who-what-when-where-how of the test, if for no other reason than to make sure you've thought of everything you need before recruiting and scheduling participants. Even the most basic tests have lots of questions that need answering: Where will you be conducting the test? How many people do you need to recruit? When will the tests take place? Who from your team is involved? What roles will people play? The list goes on. The more you have planned out, the less you have to worry about when the test happens.

For stakeholders and team members who have never participated in a usability test (or at least one organized by you) the test plan gives a good picture of what's going to happen. The logistics section is also a good place to describe a high-level agenda for the test itself—sort of an outline of the test script. If each participant is scheduled for an hour, you can describe how you break up that hour.

Another topic you might cover in the logistics section is methodology. There are several different kinds of usability testing methods, and this section is your opportunity to define and defend the one you've chosen.

Finally, your stakeholders may want to know what happens after the test, in which case you can include a description of the type of analysis you will do. Another approach is to describe the outputs—the documents and artifacts that are by-products of the test. For an informal test your outputs may be the raw notes and a short report. For more formal tests you may have additional outputs, like audio and video recordings, or more detailed reports, like an observations report and a recommendations report.
Test Scenarios

Test scenarios run from the general to the specific. For example, in some cases your scenarios may be as straightforward as "present screen to user and ask for impressions." In other cases, you may want to ask the participant to imagine himself in a specific situation and use the web site to address the situation: "You have just come down with the flu and you want to see if any of your symptoms put you at high risk."

The list of these scenarios forms the test script. Layer 2 describes a more detailed test script, but at the basic level the purpose is to give an impression of what the facilitator will ask the user to do.

Author	Message
priya Newbie Joined: 17Feb2007 Online Status: Offline Posts: 1	Topic: Creating Test Plans - Layer 1 Posted: 19Feb2007 at 1:04pm
	Creating Test Plans If you had to conduct a test tomorrow, there are three basic elements you'd need (beyond a group of users, a place to conduct the test, and—oh, yeah—a set of designs to test). The simplest test plan—described in the elements of layer 1—consists of a set of goals, a description of the logistics and method, and a set of questions to ask the participant. As stated, these are pretty broad categories, and therefore suitable for informal testing. The more formal the testing, the more details you'll need. Fleshing out the test script is described in layer 2. Further details around the logistics, described in layer 3, seem somewhat extraneous but your stakeholders may demand them. Layer 1: When You Need to Test Tomorrow The basic elements of a usability test answer three questions: What do we want out of this test, how will we conduct the test, and what will we ask users during the test? With those questions answered, you've got the barest information necessary to make a test happen. Test Objectives In planning a test it's easy to let things get out of hand. Without any direction, you'll find yourself asking users questions of increasing irrelevance. Explicitly stating a direction in the test plan, as in other documents, allows you to keep the design of the test on track. The script should be derived from the objectives and everything you ask the user should take you closer to meeting your objectives. Unfortunately, people designing usability tests frequently gloss over this part because (1) it's hard, and (2) they want to dig into the script. Consider this: Your script is meaningless unless it is working toward a specific purpose. Because the test objectives drive the discussion between the test facilitator and participant, you can make them pretty specific. Too often, the test objective looks something like this: To determine whether our web site is easy to use. You'll forgive me if I refer you to the Wikipedia entry on "duh." You can use a simple strategy to make your objectives more specific—think about why people are coming to your site. (If you created the personas described in the previous chapter, this should be easy.) By basing test objectives on user motivations, you get objectives that look more like these examples for a fictitious health encyclopedia: To determine whether people who come to the site with a specific illness in mind are satisfied with how they located relevant information. To determine whether people who are doing research on behalf of another person can easily refer someone else to the infor mation they find. To understand how people who have just been diagnosed with an illness want to revisit their research. These objectives may be used together—nothing says your test must have only one objective. Once you've written up a couple, you can do a quick litmus test. If you state the objective in the form of a question, is it an answerable question? To take the first (bad) example, the test designers could ask themselves "Is this site easy to use?" which would not be an easy question to answer. On the other hand, you could probably easily answer the question "Are people who have a specific illness in mind satisfied with how they found relevant information?" Test Logistics Your plan needs to describe the who-what-when-where-how of the test, if for no other reason than to make sure you've thought of everything you need before recruiting and scheduling participants. Even the most basic tests have lots of questions that need answering: Where will you be conducting the test? How many people do you need to recruit? When will the tests take place? Who from your team is involved? What roles will people play? The list goes on. The more you have planned out, the less you have to worry about when the test happens. For stakeholders and team members who have never participated in a usability test (or at least one organized by you) the test plan gives a good picture of what's going to happen. The logistics section is also a good place to describe a high-level agenda for the test itself—sort of an outline of the test script. If each participant is scheduled for an hour, you can describe how you break up that hour. Another topic you might cover in the logistics section is methodology. There are several different kinds of usability testing methods, and this section is your opportunity to define and defend the one you've chosen. Finally, your stakeholders may want to know what happens after the test, in which case you can include a description of the type of analysis you will do. Another approach is to describe the outputs—the documents and artifacts that are by-products of the test. For an informal test your outputs may be the raw notes and a short report. For more formal tests you may have additional outputs, like audio and video recordings, or more detailed reports, like an observations report and a recommendations report. Test Scenarios Test scenarios run from the general to the specific. For example, in some cases your scenarios may be as straightforward as "present screen to user and ask for impressions." In other cases, you may want to ask the participant to imagine himself in a specific situation and use the web site to address the situation: "You have just come down with the flu and you want to see if any of your symptoms put you at high risk." The list of these scenarios forms the test script. Layer 2 describes a more detailed test script, but at the basic level the purpose is to give an impression of what the facilitator will ask the user to do. Post Resume: Click here to Upload your Resume & Apply for Jobs

	IP Logged


One Stop Testing Forum : Software Testing @ OneStopTesting : Test Plans @ OneStopTesting