Reporting a bug is nothing but a skill

“Reporting a bug is nothing but a skill” – it is my belief. If your bug report is effective, chances are higher that it will get fixed faster. So fixing a bug depends on how effectively you report it. If test engineer is not reporting bug correctly, programmer will most likely reject this bug stating as irreproducible. This can hurt test engineers moral and sometime ego also. But I suggest, as a Software Test Engineer do not keep any type of ego like “I have reported bug correctly”, “I can reproduce it”, “Why he/she has rejected the bug?”, “It’s not my fault” etc.

What are the qualities of a good software bug report?

Anyone can write a bug report. But not everyone can write an effective bug report. You should be able to distinguish between average bug report and a good bug report. How to distinguish a good or bad bug report? It’s simple; I think you may apply following characteristics and techniques to report a bug.

• Having clearly specified bug number: Always assign a unique number to each bug report. This will help to identify the bug record. If you are using any automated bug-reporting tool then this unique number will be generated automatically each time you report the bug. Note the number and brief description of each bug you reported.
  
• Steps to Reproduce: If your bug is not reproducible it will never get fixed. You should clearly mention the steps to reproduce the bug. Do not assume or skip any reproducing step. Step by step described bug problem is easy to reproduce and fix.
• Be Specific: Do not write an essay about the problem. Be Specific and to the point. Try to summarize the problem in minimum words yet in effective way. Do not combine multiple problems even they seem to be similar. Write different reports for each problem.

How to Report a Bug?

I like to describe a simple but effective bug report format/template in below. It may vary on the bug report tool you are using. If you are writing bug report manually then some fields need to specifically mention like Bug number which should be assigned manually. Your bug report will contain:

· Reporter Description: Your name and email address.

· Product: In which product you found this bug.

· Version: The product version where you found this bug.

· Component: These are the major sub modules of the product. Sometimes it may be the form name.

· Platform: Mention the hardware platform where you found this bug. The various platforms like ‘PC’, ‘MAC’, ‘iPod’ etc.

· OS & Version: Mention the operating systems and the version where you found the bug. Operating systems like Windows, Linux, UNIX, Sun OS, Mac OS. Mention the different OS versions like Windows NT, Windows 7, Windows XP etc.

· Priority: When bug should be fixed? Priority is generally set from P1 to P5. Or “Low” to “Immediate’.

· Severity: Severity describes the impact of the bug. Types of Severity may:

o Blocker: No further testing work can be done.

o Critical: Application crash, Loss of data.

o Major: Major loss of function.

o Minor: minor loss of function.

o Trivial: Some UI enhancements.

· Status: When you are logging the bug in any bug tracking system then by default the bug status is ‘New’. Later on bug goes through various stages like Assigned, Resolved, Closed etc.

· Assign To: If you know which developer is responsible for that particular module in which bug occurred, then you can specify credential of that developer. Else keep it blank this will assign bug to module owner or manager will assign the bug to developer. Possibly add the manager email address in CC list.

· URL: Mention the page url on which bug occurred.

· Summary: A brief summary of the bug mostly in 60 or below words. Make sure your summary is reflecting what the problem is and where it is.

· Description: A detailed description of bug. Use following fields for description field:

o Reproduce steps: Clearly mention the steps to reproduce the bug.

o Expected result: How application should behave on above mentioned steps.

o Actual result: Write the actual result that will produce on running above steps.

You may also concern about the following to write a good bug report:

1) Report the problem immediately: If you find any bug while testing, do not wait to write detail bug report later. Instead write the bug report immediately. This will ensure a good and reproducible bug report. If you decide to write the bug report later on then chances are high to miss the important steps in your report.

2) Reproduce the bug three times before writing bug report: Your bug should be reproducible. Make sure your steps are robust enough to reproduce the bug without any ambiguity. If your bug is not reproducible every time you can still file a bug mentioning the periodic nature of the bug.

3) Test the same bug occurrence on other similar module: Sometimes developer use same code for different similar modules. So chances are high that bug in one module can occur in other similar modules as well. Even you can try to find more severe version of the bug you found.

4) Write a good bug summary: Bug summary will help developers to quickly analyze the bug nature. Poor quality report will unnecessarily increase the development and testing time. Communicate well through your bug report summary. Keep in mind that bug summary is used as a reference to search the bug in bug inventory.

5) Read bug report before hitting Submit button: Read all sentences, wording, steps used in bug report. See if any sentence is creating ambiguity that can lead to misinterpretation. Misleading words or sentences should be avoided in order to have a clear bug report. You musr check the spell.

6) Do not use Abusive language: It’s nice that you did a good work and found a bug but do not use this credit for criticizing developer or to attack any individual.

No doubt that your bug report should be a high quality document. Focus on writing good bug reports, spend some time on this task because this is main communication point between tester, developer and manager. Mangers should make aware to their team that writing a good bug report is primary responsibility of any tester. Your efforts towards writing good bug report will not only save company resources but also create a good relationship between you and developers.

So, for better productivity write a better bug report.

Way of Measuring Software Testing

Every step of Software Testing Process needs to be measured so as to guarantee a quality product to the customer. At the same time, measurement needs to be easy to understand and implement. Software testing process needs to follow the way of measurement:

·Size of Software: Software size means the amount of functionality of an application. The complete project estimation for testing depends on determining the size of the software. Many methods are available to compute the size software.

Widely used methods are:

o Function Point Analysis

o Use Case Estimation Methodology

· Requirement review: Software Requirement Specifications (SRS) should be obtained from the client before starting measurement of software testing process. This SRS should be:

o Complete: A complete requirements specification must precisely define all the real world situations that will be encountered and the capability's responses to them. It must not include situations that will not be encountered or unnecessary capability features.

o Consistent: A consistent specification is one where there is no conflict between individual requirement statements that define the behavior of essential capabilities and specified behavioral properties and constraints do not have an adverse impact on that behavior.

o Correct: For a requirements specification to be correct it must accurately and precisely identify the individual conditions and limitations of all situations that the desired capability will encounter and it must also define the capability's proper response to those situations.

o Structured: Related requirements should be grouped together and the document should be logically structured.

o Ranked: Requirement document should be ranked by the importance of the requirement.

o Testable: In order for a requirement specification to be testable it must be stated in such manners that pass/fail or quantitative assessment criteria can be derived from the specification itself and/or referenced information.

o Traceable: Each requirement stated within the SRS document must be uniquely identified to achieve trace ability. Uniqueness is facilitated by the use of a consistent and logical scheme for assigning identification to each specification statement within the requirements document.

o Unambiguous: A statement of a requirement is unambiguous if it can only be interpreted one way. This perhaps, is the most difficult attribute to achieve using natural language. The use of weak phrases or poor sentence structure will open the specification statement to misunderstandings.

o Validate: To validate a requirements specification all the project participants, managers, engineers and customer representatives, must be able to understand, analyze and accept or approve it.

o Verification: Requirements specification requires review and analysis by technical and operational experts in the domain addressed by the requirements. In order to be verifiable requirement specifications at one Practical Measurements for Software Testing level of abstraction must be consistent with those at another level of abstraction. Review efficiency can then be computed.

Review efficiency is a metric that provides an insight on quality of review and testing conducted.

Review efficiency = 100 * Total number of defects found by reviews / Total number of project defects
High values indicate an effective review process is implemented and defects are detected as soon as they are introduced.

· Effectiveness of testing requirements: Measuring the effectiveness of testing requirements involves;

o Specification of requirements & maintenance of Requirement Trace-ability Matrix (RTM): Specification of requirements must include;

· SRS Objective

· SRS Purpose

· Interfaces

· Functional Capabilities

· Performance Levels

· Data Structures/Elements Safety

· Reliability

· Security/Privacy

· Quality

· Constraints & limitations

Once the requirements have been specified and reviewed the next step is to update the RTM. The RTM is an extremely important document. In its simplest form, it provides a way to determine if all requirements are tested. However, the RTM can do much more. For example,

In this example, the RTM is used as a test planning tool to help determine how many tests are required, what types of tests are required, whether tests can be automated or manual, and if any existing tests can be re-used. Using the RTM in this way helps ensure that the resulting tests are most effective.

o Measuring the mapping of test cases to requirements: While we map the test cases to corresponding requirements, we must first determine the following:

· Number of requirements it tests

· Priority of requirement it tests

· Effort for its execution versus coverage of requirements

Here, the number of requirements it tests and the priority of the requirements can be obtained from the Software requirement specifications (SRS). We should see that every requirement has a set of test cases mapped to it.