Way of Measuring Software Testing

Every step of Software Testing Process needs to be measured so as to guarantee a quality product to the customer. At the same time, measurement needs to be easy to understand and implement. Software testing process needs to follow the way of measurement:

·Size of Software: Software size means the amount of functionality of an application. The complete project estimation for testing depends on determining the size of the software. Many methods are available to compute the size software.

Widely used methods are:

o Function Point Analysis

o Use Case Estimation Methodology

· Requirement review: Software Requirement Specifications (SRS) should be obtained from the client before starting measurement of software testing process. This SRS should be:

o Complete: A complete requirements specification must precisely define all the real world situations that will be encountered and the capability's responses to them. It must not include situations that will not be encountered or unnecessary capability features.

o Consistent: A consistent specification is one where there is no conflict between individual requirement statements that define the behavior of essential capabilities and specified behavioral properties and constraints do not have an adverse impact on that behavior.

o Correct: For a requirements specification to be correct it must accurately and precisely identify the individual conditions and limitations of all situations that the desired capability will encounter and it must also define the capability's proper response to those situations.

o Structured: Related requirements should be grouped together and the document should be logically structured.

o Ranked: Requirement document should be ranked by the importance of the requirement.

o Testable: In order for a requirement specification to be testable it must be stated in such manners that pass/fail or quantitative assessment criteria can be derived from the specification itself and/or referenced information.

o Traceable: Each requirement stated within the SRS document must be uniquely identified to achieve trace ability. Uniqueness is facilitated by the use of a consistent and logical scheme for assigning identification to each specification statement within the requirements document.

o Unambiguous: A statement of a requirement is unambiguous if it can only be interpreted one way. This perhaps, is the most difficult attribute to achieve using natural language. The use of weak phrases or poor sentence structure will open the specification statement to misunderstandings.

o Validate: To validate a requirements specification all the project participants, managers, engineers and customer representatives, must be able to understand, analyze and accept or approve it.

o Verification: Requirements specification requires review and analysis by technical and operational experts in the domain addressed by the requirements. In order to be verifiable requirement specifications at one Practical Measurements for Software Testing level of abstraction must be consistent with those at another level of abstraction. Review efficiency can then be computed.

Review efficiency is a metric that provides an insight on quality of review and testing conducted.

Review efficiency = 100 * Total number of defects found by reviews / Total number of project defects
High values indicate an effective review process is implemented and defects are detected as soon as they are introduced.

· Effectiveness of testing requirements: Measuring the effectiveness of testing requirements involves;

o Specification of requirements & maintenance of Requirement Trace-ability Matrix (RTM): Specification of requirements must include;

· SRS Objective

· SRS Purpose

· Interfaces

· Functional Capabilities

· Performance Levels

· Data Structures/Elements Safety

· Reliability

· Security/Privacy

· Quality

· Constraints & limitations

Once the requirements have been specified and reviewed the next step is to update the RTM. The RTM is an extremely important document. In its simplest form, it provides a way to determine if all requirements are tested. However, the RTM can do much more. For example,

In this example, the RTM is used as a test planning tool to help determine how many tests are required, what types of tests are required, whether tests can be automated or manual, and if any existing tests can be re-used. Using the RTM in this way helps ensure that the resulting tests are most effective.

o Measuring the mapping of test cases to requirements: While we map the test cases to corresponding requirements, we must first determine the following:

· Number of requirements it tests

· Priority of requirement it tests

· Effort for its execution versus coverage of requirements

Here, the number of requirements it tests and the priority of the requirements can be obtained from the Software requirement specifications (SRS). We should see that every requirement has a set of test cases mapped to it.

How can World Wide Web sites be tested?

Web sites are essentially client-server applications - with web servers and 'browser' clients. Consideration should be given to the interactions between html pages, TCP/IP communications, Internet connections, firewalls, applications that run in web pages (such as applets, javascript, plug-in applications), and applications that run on the server side (such as cgi scripts, database interfaces, logging applications, dynamic page generators, asp, etc.).

Additionally, there are a wide variety of servers and browsers, various versions of each, small but sometimes significant differences between them, variations in connection speeds, rapidly changing technologies, and multiple standards and protocols.

The end result is that testing for web sites can become a major ongoing effort.

Other considerations might include:

• What are the expected loads on the server (e.g., number of hits per unit time?), and what kind of performance is required under such loads (such as web server response time, database query response times).
• What kinds of tools will be needed for performance testing (such as web load testing tools, other tools already in house that can be adapted, web robot downloading tools, etc.)?
• Who is the target audience? What kind of browsers will they be using? What kinds of connection speeds will them by using? Are they intra- organization (thus with likely high connection speeds and similar browsers) or Internet-wide (thus with a wide variety of connection speeds and browser types)?
• What kind of performance is expected on the client side (e.g., how fast should pages appear, how fast should animations, applets, etc. load and run)?
• Will down time for server and content maintenance/upgrades be allowed? how much?
• What kinds of security (firewalls, encryptions, passwords, etc.) will be required and what is it expected to do? How can it be tested?
• How reliable are the site's Internet connections required to be? And how does that affect backup system or redundant connection requirements and testing?
• What processes will be required to manage updates to the web site's content, and what are the requirements for maintaining, tracking, and controlling page content, graphics, links, etc.?
• Which HTML specification will be adhered to? How strictly? What variations will be allowed for targeted browsers?
• Will there be any standards or requirements for page appearance and/or graphics throughout a site or parts of a site??
• How will internal and external links be validated and updated? how often?
• Can testing be done on the production system, or will a separate test system be required? How are browser caching, variations in browser option settings, dial-up connection variabilities, and real-world internet 'traffic congestion' problems to be accounted for in testing?
• How extensive or customized are the server logging and reporting requirements; are they considered an integral part of the system and do they require testing?
• How are cgi programs, applets, javascripts, ActiveX components, etc. to be maintained, tracked, controlled, and tested?
• Pages should be 3-5 screens max unless content is tightly focused on a single topic. If larger, provide internal links within the page.
• The page layouts and design elements should be consistent throughout a site, so that it's clear to the user that they're still within a site.
• Pages should be as browser-independent as possible, or pages should be provided or generated based on the browser-type.
• All pages should have links external to the page; there should be no dead-end pages.
• The page owner, revision date, and a link to a contact person or organization should be included on each page.