Topic: After the Bug Report

Message

   Imagine a customer telling you that your product is not working. You would probably have many questions:

    * How serious is the situation?
    * Is there a defect in your product, or is the defect in another part of the customer system?
    * Is the issue already known?
    * Is a solution already available?

Now imagine thousands of such incidents from around the world. How would you analyze these incidents as quickly, efficiently, and accurately as possible? After all, each report is a chance to make the product better for everyone.

This scenario is the challenge that the WebBugs team faces every day. Ideally, the team would be able to investigate every bug report in person. But this approach is not realistic, so we have designed a bug-reporting site to help us gather the information we need.

Our goal is to assist developers in creating a report that is complete and compact. A complete report allows our team's developers to reproduce the problem locally. A compact report helps us focus the investigation so that we can quickly determine the cause of the behavior.

Let's tour the bug-reporting site to see how we pursue this goal. And note that this document applies only to bugs. It does not discuss the trail for reporting documentation issues and requests for enhancements, which are other options on the bug-reporting site.
Report Elements

There are four basic parts of a report:

   1. Overview
   2. Routing information
   3. Test case
   4. Contact information

Overview

The overview provides us with a background for understanding the issue, as Table 1 indicates.
Table 1: Overview of a Bug Report
Synopsis

This becomes the report's name. A good synopsis is a unique identifier that offers a rough idea of the issue.
Description

This is a framework for understanding the problem and the test case, which you will provide later in the report. If you wish, you may include the cause of the problem and a proposed solution. Write as short a description as possible. Use a professional tone and avoid the impulse to vent your frustration. Venting is likely to distract and put off the report analyst.

Routing Information

Routing fields are the rudders that direct the report to the proper engineering group. Although we will try to correct any mistakes, filing an incorrectly classified bug report may cause a delay in the response.

Table 2 provides more detail.
Table 2: Routing Information in a Bug Report
Category and subcategory

Fill in the specific component that is exhibiting the problem. Choosing the proper category and subcategory will minimize any delay in routing the report.
Version

This is the exact release of the affected product. We ask for version information twice: The first provides a general sense of the release, and the second offers details on the exact release and build. Try to use the most up-to-date version. Sun, like most engineering organizations, focuses on the current and upcoming releases.
Regression version

If the bug is a regression, we want to know in what release the behavior worked before. This information helps to narrow the search for when and where the code changed.
Severity

This field indicates the importance of the issue. Note that for high-severity items, the submitter should seek community or paid support, because this reporting channel is not designed to help with individual cases.

Test Case:

The test case is the meat of the report. An error trace, although helpful, does not provides the richness of information available from a test case. Simply put, if we cannot reproduce the problem on our machines, we will have difficulty locating the cause of the failure and developing a solution.

As long as a report is complete, less is more. We tend to review shorter reports first, because our time is limited. Your spending a little time to make a report more concise often results in a faster response time.

Table 3 provides more detail.
Table 3: The Test Case in a Bug Report
Operating system and hardware

The platform (operating system and hardware) indicates which system will reproduce the problem. If the behavior occurs on more than one platform, pick one to remove any ambiguity.
Source code for an executable test case

The source code should be a stand-alone application that can compile without any external libraries. Again, shorter is better: Anything longer than 20 lines makes it more difficult for engineers to find the actual cause and increases the probability that the problem is actually in the bug submitter's code. Although extracting a small test case from a gigantic deployed application is a challenge, submit as short a test case as possible. This exercise may reveal that the problem is in one's own code, which means that a solution is immediately available.
Steps to reproduce

Include the command-line invocation, even if it seems obvious.
Expected result

The expected behavior tells us how the test case should behave if no bug were present. If the test case produces this expected result, we know that a fix works.
Actual result

The actual behavior tells us the current results of the test case. If Sun engineering does not see this exact behavior, then the system configuration may be causing the problem. 

Contact Information

The email address allows us to communicate with the report submitter in case we want to clarify anything or send an update. The other information helps us to be more intelligent and polite in our responses.

Table 4 provides more detail.
Table 4: Contact Information in a Bug Report
Your name

This field tells us how to politely address the submitter.
Company

The company name provides us with more background about the submitter.
Email

An email address is critical because it is the only way to contact the submitter. The willingness to communicate indicates the submitter's seriousness, so we delete bug reports in which the submitter has not provided a valid email address.
Sun online account ID

This information identifies submitters who are part of the Sun Developer Network.

Examples

Here are links to two reports for the same issue: imprecision when multiplying two floating point numbers. Note how the second version makes analysis much easier.

    * This long report is vague yet complicated. In addition, the submitter is rude. What reasonable engineer would spend time sifting through the report?
    * This model report is excellent. Because it is complete and compact, identifying the ultimate fault will be easier. 

By the way, the imprecision that both reports point to is expected. The Java programming language, like most programming languages, uses binary numbers to represent our base-10 numbering system. As a result, standard floating-point arithmetic in the Java language is not exact. 
Conclusion

Writing a complete, compact report takes skill and craftsmanship. We hope that our discussion of report elements and sample reports has made this task easier for you. The resulting improvements in reports will mean faster fixes and a higher-quality final product. Ultimately, we all will benefit.

Edited by moderator - 06May2007 at 11:00pm

Author	Message
Pallavi Newbie Joined: 23Feb2007 Online Status: Offline Posts: 1	Topic: After the Bug Report Posted: 23Feb2007 at 9:15am
	Imagine a customer telling you that your product is not working. You would probably have many questions: * How serious is the situation? * Is there a defect in your product, or is the defect in another part of the customer system? * Is the issue already known? * Is a solution already available? Now imagine thousands of such incidents from around the world. How would you analyze these incidents as quickly, efficiently, and accurately as possible? After all, each report is a chance to make the product better for everyone. This scenario is the challenge that the WebBugs team faces every day. Ideally, the team would be able to investigate every bug report in person. But this approach is not realistic, so we have designed a bug-reporting site to help us gather the information we need. Our goal is to assist developers in creating a report that is complete and compact. A complete report allows our team's developers to reproduce the problem locally. A compact report helps us focus the investigation so that we can quickly determine the cause of the behavior. Let's tour the bug-reporting site to see how we pursue this goal. And note that this document applies only to bugs. It does not discuss the trail for reporting documentation issues and requests for enhancements, which are other options on the bug-reporting site. Report Elements There are four basic parts of a report: 1. Overview 2. Routing information 3. Test case 4. Contact information Overview The overview provides us with a background for understanding the issue, as Table 1 indicates. Table 1: Overview of a Bug Report Synopsis This becomes the report's name. A good synopsis is a unique identifier that offers a rough idea of the issue. Description This is a framework for understanding the problem and the test case, which you will provide later in the report. If you wish, you may include the cause of the problem and a proposed solution. Write as short a description as possible. Use a professional tone and avoid the impulse to vent your frustration. Venting is likely to distract and put off the report analyst. Routing Information Routing fields are the rudders that direct the report to the proper engineering group. Although we will try to correct any mistakes, filing an incorrectly classified bug report may cause a delay in the response. Table 2 provides more detail. Table 2: Routing Information in a Bug Report Category and subcategory Fill in the specific component that is exhibiting the problem. Choosing the proper category and subcategory will minimize any delay in routing the report. Version This is the exact release of the affected product. We ask for version information twice: The first provides a general sense of the release, and the second offers details on the exact release and build. Try to use the most up-to-date version. Sun, like most engineering organizations, focuses on the current and upcoming releases. Regression version If the bug is a regression, we want to know in what release the behavior worked before. This information helps to narrow the search for when and where the code changed. Severity This field indicates the importance of the issue. Note that for high-severity items, the submitter should seek community or paid support, because this reporting channel is not designed to help with individual cases. Test Case: The test case is the meat of the report. An error trace, although helpful, does not provides the richness of information available from a test case. Simply put, if we cannot reproduce the problem on our machines, we will have difficulty locating the cause of the failure and developing a solution. As long as a report is complete, less is more. We tend to review shorter reports first, because our time is limited. Your spending a little time to make a report more concise often results in a faster response time. Table 3 provides more detail. Table 3: The Test Case in a Bug Report Operating system and hardware The platform (operating system and hardware) indicates which system will reproduce the problem. If the behavior occurs on more than one platform, pick one to remove any ambiguity. Source code for an executable test case The source code should be a stand-alone application that can compile without any external libraries. Again, shorter is better: Anything longer than 20 lines makes it more difficult for engineers to find the actual cause and increases the probability that the problem is actually in the bug submitter's code. Although extracting a small test case from a gigantic deployed application is a challenge, submit as short a test case as possible. This exercise may reveal that the problem is in one's own code, which means that a solution is immediately available. Steps to reproduce Include the command-line invocation, even if it seems obvious. Expected result The expected behavior tells us how the test case should behave if no bug were present. If the test case produces this expected result, we know that a fix works. Actual result The actual behavior tells us the current results of the test case. If Sun engineering does not see this exact behavior, then the system configuration may be causing the problem. Contact Information The email address allows us to communicate with the report submitter in case we want to clarify anything or send an update. The other information helps us to be more intelligent and polite in our responses. Table 4 provides more detail. Table 4: Contact Information in a Bug Report Your name This field tells us how to politely address the submitter. Company The company name provides us with more background about the submitter. Email An email address is critical because it is the only way to contact the submitter. The willingness to communicate indicates the submitter's seriousness, so we delete bug reports in which the submitter has not provided a valid email address. Sun online account ID This information identifies submitters who are part of the Sun Developer Network. Examples Here are links to two reports for the same issue: imprecision when multiplying two floating point numbers. Note how the second version makes analysis much easier. * This long report is vague yet complicated. In addition, the submitter is rude. What reasonable engineer would spend time sifting through the report? * This model report is excellent. Because it is complete and compact, identifying the ultimate fault will be easier. By the way, the imprecision that both reports point to is expected. The Java programming language, like most programming languages, uses binary numbers to represent our base-10 numbering system. As a result, standard floating-point arithmetic in the Java language is not exact. Conclusion Writing a complete, compact report takes skill and craftsmanship. We hope that our discussion of report elements and sample reports has made this task easier for you. The resulting improvements in reports will mean faster fixes and a higher-quality final product. Ultimately, we all will benefit. Edited by moderator - 06May2007 at 11:00pm Post Resume: Click here to Upload your Resume & Apply for Jobs

	IP Logged


One Stop Testing Forum : Software Testing @ OneStopTesting : Bug Report @ OneStopTesting