Topic: Preparing a Test Plan

Message

Preparing a Test Plan: The Basics

In simplest terms, the function of a test plan is to tell people what to do in anticipation of and during a usability test. It is a set of instructions, grounded in project-specific objectives. Like any document in this book, the test plan will be shaped by a situation analysis—an assessment of the purpose, timing, and audience for the document.
The Why and When of Usability Testing

For most other documents in this book, the situation analysis calls for assessing the purpose of the document itself. In this case, the question isn't "Why should I do a test plan?" as it might be with wireframes or personas. Instead, the more relevant question is "Why should I do usability testing at all?" because if you're doing a usability test, you need to do a test plan. Like any good business meeting, you need to go in prepared, and a test plan gives you that preparation.

Still, the answer to that question—"Why do usability testing?"—will define the content of the test plan. Usability testing is an essential component of design, but it can't be done for the sake of itself. To understand how the purpose of the test affects the content of the test plan, we need to look at the two main kinds of testing: formative and summative. Formative tests take place during the design process, allowing the design team to do research on early versions of a design as it's taking shape. Summative tests occur at the end of the design process, with a more or less final product.

Plans for formative tests: You'll find that most formative testing is to assess the usability of a specific part of a web site—the home page and high-level pages, the checkout process, or the navigation labeling. The test plan needs to recognize that participants will only experience part of the entire interface. It may be more rigidly scripted to focus on specific issues and avoid users' missteps into unfinished parts of the interface.

Plans for summative tests: When conducted with a final product, usability testing can be more open ended. Rather than identifying particular tasks, your script may focus more on the post-test questions to glean the participant's opinion of the product.
Adjusting the Test Plan for Your Audience

The two main audiences for usability test plans are the project stakeholders and the test facilitator—the person doing most of the orchestration during the test. There may be other players in the test—people taking notes, people managing the facility, people working the recording equipment—or there may be one person who does it all. The facilitator may be someone on your team (i.e., you) or it may be a contractor or freelancer you've hired for this purpose. In the latter case, the script needs to be as specific as possible.

Frankly, even if you're doing the test yourself, making the script specific can be very helpful. During the test you'll be thinking about so many things at the same time—what the participant is doing right then, whether the recording devices are working, what will happen when the user clicks a button or link—that you won't want to have to worry about what to do next.

A specific script is also useful for stakeholders. By being explicit about what the facilitator will say to the participants, the script can help set the stakeholders' expectations, especially if they've never been involved with usability testing.

If the facilitator is outside your project team—a contractor you brought in just for tests, for example—your test plan may need to be explicit in other areas as well, like the purpose of the test, the test procedure, and the project background. It's easy to take this information for granted when you're working with the same set of people throughout the process. But for a new or unfamiliar team, a fully fleshed-out test plan can help build a common understanding of the project, the methodology, and how they fit together.
Making Test Plans Unboring

The situation analysis—understanding the who, when, and why of usability testing—helps define the contents of the test, providing direction for what kinds of information you need to provide and how much detail to give. With these decisions made, you'll need to think about how to format the document itself. Although this isn't a document heavy on the visuals, there are still several considerations for formatting.

Prose or bullets: Items like test objectives, background, and method can be described in straight paragraphs or through bullet points. The decision is more than aesthetic. The primary consideration is your audience, since you can communicate the same message with either bullets or prose. Bullets will be more effective if you do not need to go into a lot of detail. The objectives for the imaginary test at the beginning of this chapter might be rendered in prose like this:

Users researching health information online are a committed and persistent audience. Despite their keen interest—or maybe because of it—they are especially picky about the web sites they'll use to find information. A web site should not just have the most up-to-date information. It needs to be easy to find, written well, and in a format that gives users lots of flexibility in how they use it. The purpose of this usability test is to see how well our site lives up to the expectations of our users, by putting potential users in mock scenarios and observing the effectiveness of the web site design.

Specifically, we want to learn whether people with a specific illness can quickly find useful information. They should be able to locate it quickly and it should be compelling enough to make them want to stick around. This test will also measure whether the site makes it easy to refer other people to information—our users typically are doing research on someone else's behalf. Finally, we recognize that our users don't come to the site once. They return again and again to learn more or to focus on a new aspect of their condition. This test will identify how the design can be improved to more effectively meet this need.

Distinguishing "say aloud" from instruction: There are two types of information in the script—things to expose to the participant and things that give the facilitator background on a particular task or scenario. In some cases, the facilitator needs to read things out loud to the participant; in others, the participant must read to him- or herself. Figure 3.4 illustrates all the different parts of a usability script.

Author	Message
Amrita Groupie Joined: 13Feb2007 Online Status: Offline Posts: 57	Topic: Preparing a Test Plan Posted: 19Feb2007 at 2:14pm
	Preparing a Test Plan: The Basics In simplest terms, the function of a test plan is to tell people what to do in anticipation of and during a usability test. It is a set of instructions, grounded in project-specific objectives. Like any document in this book, the test plan will be shaped by a situation analysis—an assessment of the purpose, timing, and audience for the document. The Why and When of Usability Testing For most other documents in this book, the situation analysis calls for assessing the purpose of the document itself. In this case, the question isn't "Why should I do a test plan?" as it might be with wireframes or personas. Instead, the more relevant question is "Why should I do usability testing at all?" because if you're doing a usability test, you need to do a test plan. Like any good business meeting, you need to go in prepared, and a test plan gives you that preparation. Still, the answer to that question—"Why do usability testing?"—will define the content of the test plan. Usability testing is an essential component of design, but it can't be done for the sake of itself. To understand how the purpose of the test affects the content of the test plan, we need to look at the two main kinds of testing: formative and summative. Formative tests take place during the design process, allowing the design team to do research on early versions of a design as it's taking shape. Summative tests occur at the end of the design process, with a more or less final product. Plans for formative tests: You'll find that most formative testing is to assess the usability of a specific part of a web site—the home page and high-level pages, the checkout process, or the navigation labeling. The test plan needs to recognize that participants will only experience part of the entire interface. It may be more rigidly scripted to focus on specific issues and avoid users' missteps into unfinished parts of the interface. Plans for summative tests: When conducted with a final product, usability testing can be more open ended. Rather than identifying particular tasks, your script may focus more on the post-test questions to glean the participant's opinion of the product. Adjusting the Test Plan for Your Audience The two main audiences for usability test plans are the project stakeholders and the test facilitator—the person doing most of the orchestration during the test. There may be other players in the test—people taking notes, people managing the facility, people working the recording equipment—or there may be one person who does it all. The facilitator may be someone on your team (i.e., you) or it may be a contractor or freelancer you've hired for this purpose. In the latter case, the script needs to be as specific as possible. Frankly, even if you're doing the test yourself, making the script specific can be very helpful. During the test you'll be thinking about so many things at the same time—what the participant is doing right then, whether the recording devices are working, what will happen when the user clicks a button or link—that you won't want to have to worry about what to do next. A specific script is also useful for stakeholders. By being explicit about what the facilitator will say to the participants, the script can help set the stakeholders' expectations, especially if they've never been involved with usability testing. If the facilitator is outside your project team—a contractor you brought in just for tests, for example—your test plan may need to be explicit in other areas as well, like the purpose of the test, the test procedure, and the project background. It's easy to take this information for granted when you're working with the same set of people throughout the process. But for a new or unfamiliar team, a fully fleshed-out test plan can help build a common understanding of the project, the methodology, and how they fit together. Making Test Plans Unboring The situation analysis—understanding the who, when, and why of usability testing—helps define the contents of the test, providing direction for what kinds of information you need to provide and how much detail to give. With these decisions made, you'll need to think about how to format the document itself. Although this isn't a document heavy on the visuals, there are still several considerations for formatting. Prose or bullets: Items like test objectives, background, and method can be described in straight paragraphs or through bullet points. The decision is more than aesthetic. The primary consideration is your audience, since you can communicate the same message with either bullets or prose. Bullets will be more effective if you do not need to go into a lot of detail. The objectives for the imaginary test at the beginning of this chapter might be rendered in prose like this: Users researching health information online are a committed and persistent audience. Despite their keen interest—or maybe because of it—they are especially picky about the web sites they'll use to find information. A web site should not just have the most up-to-date information. It needs to be easy to find, written well, and in a format that gives users lots of flexibility in how they use it. The purpose of this usability test is to see how well our site lives up to the expectations of our users, by putting potential users in mock scenarios and observing the effectiveness of the web site design. Specifically, we want to learn whether people with a specific illness can quickly find useful information. They should be able to locate it quickly and it should be compelling enough to make them want to stick around. This test will also measure whether the site makes it easy to refer other people to information—our users typically are doing research on someone else's behalf. Finally, we recognize that our users don't come to the site once. They return again and again to learn more or to focus on a new aspect of their condition. This test will identify how the design can be improved to more effectively meet this need. Distinguishing "say aloud" from instruction: There are two types of information in the script—things to expose to the participant and things that give the facilitator background on a particular task or scenario. In some cases, the facilitator needs to read things out loud to the participant; in others, the participant must read to him- or herself. Figure 3.4 illustrates all the different parts of a usability script. Post Resume: Click here to Upload your Resume & Apply for Jobs

	IP Logged


One Stop Testing Forum : Software Testing @ OneStopTesting : Test Plans @ OneStopTesting