Topic: Test plans in software development

Message

Test plans in software development

    In software testing, a
test plan gives detailed testing information regarding an upcoming
testing effort, including

    * Scope of testing

    * Schedule

    * Test Deliverables

    * Release Criteria

    * Risks and Contingencies

  Test plan template, IEEE 829 format

   1. Test Plan Identifier (TPI)

   2. References

   3. Introduction

   4. Test Items

   5. Software Risk Issues

   6. Features to be Tested

   7. Features not to be Tested

   8. Approach

   9. Item Pass/Fail Criteria

  10. Entry & Exit Criteria

  11. Suspension Criteria and Resumption Requirements

  12. Test Deliverables

  13. Remaining Test Tasks

  14. Environmental Needs

  15. Staffing and Training Needs

  16. Responsibilities

  17. Schedule

  18. Planning Risks and Contingencies

  19. Approvals

  20. Glossary

  Test plan identifier

Master test plan for the Line of Credit Payment System. TP_PA1.0

[edit] References

List all documents that support this test plan.

Documents that are referenced include:

    * Project Plan

    * System Requirements specifications.

    * High Level design document.

    * Detail design document.

    * Development and Test process standards.

    * Methodology guidelines and examples.

    * Corporate standards and guidelines.

  Introduction

State
the purpose of the Plan, possibly identifying the level of the plan
(master etc.). This is essentially the executive summary part of the
plan.

You may want to include any references to other plans,
documents or items that contain information relevant to this
project/process.

Identify the objective of the plan or scope of the plan in relation to the Software Project
plan that it relates to. Other items may include, resource and budget
constraints, scope of the testing effort, how testing relates to other
evaluation activities (Analysis & Reviews), and possible the
process to be used for change control and communication and
coordination of key activities.

As this is the "Executive Summary" keep information brief and to the point.

 Test items (functions)

These
are things you intend to test within the scope of this test plan.
Essentially, something you will test, a list of what is to be tested.
This can be developed from the software application inventories as well
as other sources of documentation and information.

This can be controlled on a local Configuration Management
(CM) process if you have one. This information includes version
numbers, configuration requirements where needed, (especially if
multiple versions of the product are supported). It may also include
key delivery schedule issues for critical elements.

Remember, what you are testing is what you intend to deliver to the Client.

This
section can be oriented to the level of the test plan. For higher
levels it may be by application or functional area, for lower levels it
may be by program, unit, module or build.

  Software risk issues

Identify what software is to be tested and what the critical areas are, such as:

   1. Delivery of a third party product.

   2. New version of interfacing software.

   3. Ability to use and understand a new package/tool, etc.

   4. Extremely complex functions.

   5. Modifications to components with a past history of failure.

   6. Poorly documented modules or change requests.

There are some inherent software risks such as complexity; these need to be identified.

   1. Safety.

   2. Multiple interfaces.

   3. Impacts on Client.

   4. Government regulations and rules.

Another
key area of risk is a misunderstanding of the original requirements.
This can occur at the management, user and developer levels. Be aware
of vague or unclear requirements and requirements that cannot be tested.

The
past history of defects (bugs) discovered during Unit testing will help
identify potential areas within the software that are risky. If the
unit testing discovered a large number of defects or a tendency towards
defects in a particular area of the software, this is an indication of
potential future problems. It is the nature of defects to cluster and
clump together. If it was defect ridden earlier, it will most likely
continue to be defect prone.

One good approach to define where the risks are is to have several brainstorming sessions.

    * Start with ideas, such as, what worries me about this project/application.

 Features to be tested

This
is a listing of what is to be tested from the user's viewpoint of what
the system does. This is not a technical description of the software,
but a USERS view of the functions.

Set the level of risk for
each feature. Use a simple rating scale such as (H, M, L): High, Medium
and Low. These types of levels are understandable to a User. You should
be prepared to discuss why a particular level was chosen.

Sections
4 and 6 are very similar, and the only true difference is the point of
view. Section 4 is a technical type description including version
numbers and other technical information and Section 6 is from the
User’s viewpoint. Users do not understand technical software
terminology; they understand functions and processes as they relate to
their jobs.

  Features not to be tested

This is a listing
of what is 'not' to be tested from both the user's viewpoint of what
the system does and a configuration management/version control view.
This is not a technical description of the software, but a user's view
of the functions.

Identify why the feature is not to be tested, there can be any number of reasons.

    * Not to be included in this release of the Software.

    * Low risk, has been used before and was considered stable.

* Will be released but not tested or documented as a functional part of
the release of this version of the software.

Sections
6 and 7 are directly related to Sections 5 and 17. What will and will
not be tested are directly affected by the levels of acceptable risk
within the project, and what does not get tested affects the level of
risk of the project.

  Approach (strategy)

This is your
overall test strategy for this test plan; it should be appropriate to
the level of the plan (master, acceptance, etc.) and should be in
agreement with all higher and lower levels of plans. Overall rules and
processes should be identified.

    * Are any special tools to be used and what are they?

    * Will the tool require special training?

    * What metrics will be collected?

    * Which level is each metric to be collected at?

    * How is Configuration Management to be handled?

    * How many different configurations will be tested?

    * Hardware

    * Software

    * Combinations of HW, SW and other vendor packages

    * What levels of regression testing will be done and how much at each test level?

    * Will regression testing be based on severity of defects detected?

    * How will elements in the requirements and design that do not make sense or are untestable be processed?

If this is a master test plan the overall project testing approach and coverage requirements must also be identified.

Specify if there are special requirements for the testing.

    * Only the full component will be tested.

    * A specified segment of grouping of features/components must be tested together.

Other information that may be useful in setting the approach are:

* MTBF, Mean Time Between Failures - if this is a valid measurement for
the test involved and if the data is available.

    * SRE, Software Reliability Engineering - if this methodology is in use and if the information is available.

How will meetings and other organizational processes be handled?

[edit] Item pass/fail criteria

Show stopper issues. Specify the criteria to be used to determine whether each test item has passed or failed.

  Entry & exit criteria

Specify the criteria to be used to start testing and how you know when to stop the testing process.

 Suspension criteria & resumption requirements

Suspension
criteria specifies the criteria to be used to suspend all or a portion
of the testing activities while resumption criteria specifies when
testing can resume after it has been suspended.

  Test deliverables

List
documents, reports, charts, that will be presented to stakeholders on a
regular basis during testing and when testing has been completed.

  Remaining test tasks

If
this is a multi-phase process or if the application is to be released
in increments there may be parts of the application that this plan does
not address. These areas need to be identified to avoid any confusion
should defects be reported back on those future functions. This will
also allow the users and testers to avoid incomplete functions and
prevent waste of resources chasing non-defects.

If the project
is being developed as a multi-party process, this plan may only cover a
portion of the total functions/features. This status needs to be
identified so that those other areas have plans developed for them and
to avoid wasting resources tracking defects that do not relate to this
plan.

When a third party is developing the software, this
section may contain descriptions of those test tasks belonging to both
the internal groups and the external groups.

  Environmental needs

Are there any special requirements for this test plan, such as:

    * Special hardware such as simulators, static generators etc.

* How will test data be provided. Are there special collection
requirements or specific ranges of data that must be provided?

    * How much testing will be done on each component of a multi-part feature?

    * Special power requirements.

    * Specific versions of other supporting software.

    * Restricted use of the system during testing.

  Staffing and training needs

Training on the application/system.

Training for any test tools to be used.

The
Test Items and Responsibilities sections affect this section. What is
to be tested and who is responsible for the testing and training.

 Responsibilities

Who is in charge?

This issue includes all areas of the plan. Here are some examples:

    * Setting risks.

    * Selecting features to be tested and not tested.

    * Setting overall strategy for this level of plan.

    * Ensuring all required elements are in place for testing.

    * Providing for resolution of scheduling conflicts, especially, if testing is done on the production system.

    * Who provides the required training?

    * Who makes the critical go/no go decisions for items not covered in the test plans?

  Planning risks and contingencies

What are the overall risks to the project with an emphasis on the testing process?

    * Lack of personnel resources when testing is to begin.

    * Lack of availability of required hardware, software, data or tools.

    * Late delivery of the software, hardware or tools.

    * Delays in training on the application and/or tools.

    * Changes to the original requirements or designs.

    * Complexities involved in testing the applications

Specify what will be done for various events, for example:

Requirements
definition will be complete by January 1, 20XX, and, if the
requirements change after that date, the following actions will be
taken:

    * The test schedule and development schedule will
move out an appropriate number of days. This rarely occurs, as most
projects tend to have fixed delivery dates.

    * The number of tests performed will be reduced.

    * The number of acceptable defects will be increased.

o These two items could lower the overall quality of the delivered
product.

    * Resources will be added to the test team.

    * The test team will work overtime (this could affect team morale).

    * The scope of the plan may be changed.

    * There may be some optimisation of resources. This should be avoided, if possible, for obvious reasons.

Management is usually reluctant to accept scenarios such as the one above even though they have seen it happen in the past.

The
important thing to remember is that, if you do nothing at all, the
usual result is that testing is cut back or omitted completely, neither
of which should be an acceptable option.

Edited by moderator - 26Apr2007 at 11:44pm

Author	Message
Anamica Newbie Joined: 05Apr2007 Online Status: Offline Posts: 21	Topic: Test plans in software development Posted: 05Apr2007 at 3:32am
	Test plans in software development In software testing, a test plan gives detailed testing information regarding an upcoming testing effort, including * Scope of testing * Schedule * Test Deliverables * Release Criteria * Risks and Contingencies Test plan template, IEEE 829 format 1. Test Plan Identifier (TPI) 2. References 3. Introduction 4. Test Items 5. Software Risk Issues 6. Features to be Tested 7. Features not to be Tested 8. Approach 9. Item Pass/Fail Criteria 10. Entry & Exit Criteria 11. Suspension Criteria and Resumption Requirements 12. Test Deliverables 13. Remaining Test Tasks 14. Environmental Needs 15. Staffing and Training Needs 16. Responsibilities 17. Schedule 18. Planning Risks and Contingencies 19. Approvals 20. Glossary Test plan identifier Master test plan for the Line of Credit Payment System. TP_PA1.0 [edit] References List all documents that support this test plan. Documents that are referenced include: * Project Plan * System Requirements specifications. * High Level design document. * Detail design document. * Development and Test process standards. * Methodology guidelines and examples. * Corporate standards and guidelines. Introduction State the purpose of the Plan, possibly identifying the level of the plan (master etc.). This is essentially the executive summary part of the plan. You may want to include any references to other plans, documents or items that contain information relevant to this project/process. Identify the objective of the plan or scope of the plan in relation to the Software Project plan that it relates to. Other items may include, resource and budget constraints, scope of the testing effort, how testing relates to other evaluation activities (Analysis & Reviews), and possible the process to be used for change control and communication and coordination of key activities. As this is the "Executive Summary" keep information brief and to the point. Test items (functions) These are things you intend to test within the scope of this test plan. Essentially, something you will test, a list of what is to be tested. This can be developed from the software application inventories as well as other sources of documentation and information. This can be controlled on a local Configuration Management (CM) process if you have one. This information includes version numbers, configuration requirements where needed, (especially if multiple versions of the product are supported). It may also include key delivery schedule issues for critical elements. Remember, what you are testing is what you intend to deliver to the Client. This section can be oriented to the level of the test plan. For higher levels it may be by application or functional area, for lower levels it may be by program, unit, module or build. Software risk issues Identify what software is to be tested and what the critical areas are, such as: 1. Delivery of a third party product. 2. New version of interfacing software. 3. Ability to use and understand a new package/tool, etc. 4. Extremely complex functions. 5. Modifications to components with a past history of failure. 6. Poorly documented modules or change requests. There are some inherent software risks such as complexity; these need to be identified. 1. Safety. 2. Multiple interfaces. 3. Impacts on Client. 4. Government regulations and rules. Another key area of risk is a misunderstanding of the original requirements. This can occur at the management, user and developer levels. Be aware of vague or unclear requirements and requirements that cannot be tested. The past history of defects (bugs) discovered during Unit testing will help identify potential areas within the software that are risky. If the unit testing discovered a large number of defects or a tendency towards defects in a particular area of the software, this is an indication of potential future problems. It is the nature of defects to cluster and clump together. If it was defect ridden earlier, it will most likely continue to be defect prone. One good approach to define where the risks are is to have several brainstorming sessions. * Start with ideas, such as, what worries me about this project/application. Features to be tested This is a listing of what is to be tested from the user's viewpoint of what the system does. This is not a technical description of the software, but a USERS view of the functions. Set the level of risk for each feature. Use a simple rating scale such as (H, M, L): High, Medium and Low. These types of levels are understandable to a User. You should be prepared to discuss why a particular level was chosen. Sections 4 and 6 are very similar, and the only true difference is the point of view. Section 4 is a technical type description including version numbers and other technical information and Section 6 is from the User’s viewpoint. Users do not understand technical software terminology; they understand functions and processes as they relate to their jobs. Features not to be tested This is a listing of what is 'not' to be tested from both the user's viewpoint of what the system does and a configuration management/version control view. This is not a technical description of the software, but a user's view of the functions. Identify why the feature is not to be tested, there can be any number of reasons. * Not to be included in this release of the Software. * Low risk, has been used before and was considered stable. * Will be released but not tested or documented as a functional part of the release of this version of the software. Sections 6 and 7 are directly related to Sections 5 and 17. What will and will not be tested are directly affected by the levels of acceptable risk within the project, and what does not get tested affects the level of risk of the project. Approach (strategy) This is your overall test strategy for this test plan; it should be appropriate to the level of the plan (master, acceptance, etc.) and should be in agreement with all higher and lower levels of plans. Overall rules and processes should be identified. * Are any special tools to be used and what are they? * Will the tool require special training? * What metrics will be collected? * Which level is each metric to be collected at? * How is Configuration Management to be handled? * How many different configurations will be tested? * Hardware * Software * Combinations of HW, SW and other vendor packages * What levels of regression testing will be done and how much at each test level? * Will regression testing be based on severity of defects detected? * How will elements in the requirements and design that do not make sense or are untestable be processed? If this is a master test plan the overall project testing approach and coverage requirements must also be identified. Specify if there are special requirements for the testing. * Only the full component will be tested. * A specified segment of grouping of features/components must be tested together. Other information that may be useful in setting the approach are: * MTBF, Mean Time Between Failures - if this is a valid measurement for the test involved and if the data is available. * SRE, Software Reliability Engineering - if this methodology is in use and if the information is available. How will meetings and other organizational processes be handled? [edit] Item pass/fail criteria Show stopper issues. Specify the criteria to be used to determine whether each test item has passed or failed. Entry & exit criteria Specify the criteria to be used to start testing and how you know when to stop the testing process. Suspension criteria & resumption requirements Suspension criteria specifies the criteria to be used to suspend all or a portion of the testing activities while resumption criteria specifies when testing can resume after it has been suspended. Test deliverables List documents, reports, charts, that will be presented to stakeholders on a regular basis during testing and when testing has been completed. Remaining test tasks If this is a multi-phase process or if the application is to be released in increments there may be parts of the application that this plan does not address. These areas need to be identified to avoid any confusion should defects be reported back on those future functions. This will also allow the users and testers to avoid incomplete functions and prevent waste of resources chasing non-defects. If the project is being developed as a multi-party process, this plan may only cover a portion of the total functions/features. This status needs to be identified so that those other areas have plans developed for them and to avoid wasting resources tracking defects that do not relate to this plan. When a third party is developing the software, this section may contain descriptions of those test tasks belonging to both the internal groups and the external groups. Environmental needs Are there any special requirements for this test plan, such as: * Special hardware such as simulators, static generators etc. * How will test data be provided. Are there special collection requirements or specific ranges of data that must be provided? * How much testing will be done on each component of a multi-part feature? * Special power requirements. * Specific versions of other supporting software. * Restricted use of the system during testing. Staffing and training needs Training on the application/system. Training for any test tools to be used. The Test Items and Responsibilities sections affect this section. What is to be tested and who is responsible for the testing and training. Responsibilities Who is in charge? This issue includes all areas of the plan. Here are some examples: * Setting risks. * Selecting features to be tested and not tested. * Setting overall strategy for this level of plan. * Ensuring all required elements are in place for testing. * Providing for resolution of scheduling conflicts, especially, if testing is done on the production system. * Who provides the required training? * Who makes the critical go/no go decisions for items not covered in the test plans? Planning risks and contingencies What are the overall risks to the project with an emphasis on the testing process? * Lack of personnel resources when testing is to begin. * Lack of availability of required hardware, software, data or tools. * Late delivery of the software, hardware or tools. * Delays in training on the application and/or tools. * Changes to the original requirements or designs. * Complexities involved in testing the applications Specify what will be done for various events, for example: Requirements definition will be complete by January 1, 20XX, and, if the requirements change after that date, the following actions will be taken: * The test schedule and development schedule will move out an appropriate number of days. This rarely occurs, as most projects tend to have fixed delivery dates. * The number of tests performed will be reduced. * The number of acceptable defects will be increased. o These two items could lower the overall quality of the delivered product. * Resources will be added to the test team. * The test team will work overtime (this could affect team morale). * The scope of the plan may be changed. * There may be some optimisation of resources. This should be avoided, if possible, for obvious reasons. Management is usually reluctant to accept scenarios such as the one above even though they have seen it happen in the past. The important thing to remember is that, if you do nothing at all, the usual result is that testing is cut back or omitted completely, neither of which should be an acceptable option. Edited by moderator - 26Apr2007 at 11:44pm Post Resume: Click here to Upload your Resume & Apply for Jobs

	IP Logged


One Stop Testing Forum : Software Testing @ OneStopTesting : Test Plans @ OneStopTesting