Some misconceptions about agile testing

Agile development is become popular to organizations day by day. Testing persons needs to perform ‘Agile Testing’ for agile development. There are lots of misconceptions about agile testing. Some very common myths are discussed below:

Myth1: Agile Testing is ad hoc

Ad hoc testing is performed without planning and documentation. Agile involves with planning sprints and resources forward of time. The test cycles are sync with development cycles; that are an inherent part of the planning.  The agile test plans are based on the user stories of a sprint. The stories may be included or removed from a sprint. Agile testing is centered on the developed stories; so, it is not ad hoc.

Myth2: Testing in Agile is undocumented, quick and dirty

Documentation is an important part of agile project. Agile insists on rapid communication. The only difference with other mythologies is that agile gives more important to face-to-face interaction than extreme dependence on written communication. Agile maintains tools for communication.  

The agile teams are usually provided with guidelines and requirements on the level of documentation and coding comments that they need to incorporate.

Myth3: Agile Testing does not have defined strategies

Testing cycle is synchronized with development, and it acts as quality gateways for the sprints in agile project. Multiple approaches are followed to quickly test the stories of a sprint. Also, agile testing helps in continuous monitoring of the application after release; and ensuring a complete feedback loop for the next sprints.

Myth4: Agile testing is tool oriented

Agile is a person oriented methodology. Agile gives more importance to person over process and tools. Efficient development and testing practices are more important than just being able to use tools. Test teams are engaged with daily sprint meetings from the start of user story identification. So, it helps to identify issues on early in the sprint.

Myth5: Agile Testing is best done by developers

Testing is inspired through the agile methodology. It is done by both developers and testers. It is a collaborative work approach between developers and testers. Testing teams are seamlessly integrated with the development life-cycle. The role of a tester in agile becomes more proactive.  

Testers seamlessly collaborate with the developers to meet their goal of delivering a bug free product by finding the bugs, fixing those, and verifying those before a sprint release.

Myth6: Agile Testing should all be at same geographical location

It is definitely good to have the whole team in one geographical location. Teams are assembled from different geographical locations with the help of technology in a global company. Meetings can be held involving people all around the world at the click of a button. There are some tools available for collaboration and file sharing.

Testing teams should be ready to accommodate different time zones and leverage technology to adopt efficient meeting practices. With a little effort, flexibility, adaptability and the proper mind-set, distributed teams can just be as agile as the co-located teams.

So, try to avoid the above listed misconceptions and start Agile from today.

What & How to test a Database – The Basics

Today I’ll discuss about some basic techniques of the database testing. A Test Engineer needs to test DB on the testing phase of software. Database is the most important part of any software. It is impossible for us to develop software without Database. It does not matter at all whether it is web or desktop, client server or peer to peer, enterprise or individual business, database is working at back-end.

Currently, several database tools are available in the market like MS-Access, MS SQL Server, Oracle, MySQL, PostgreSQL, DB2 etc.  All of these vary in cost, robustness, features and security. Each of these DBs possesses its own benefits and drawbacks. One thing is certain; a business application must be built using one of these or other DB Tools.

Before I start the What & How topic, I would like to share some common characteristics of Database. The end user mainly operates the CRUD (Create, Retrieve, Update, and Delete) operations when the application is under execution. Every Test Engineer needs to know at least the basic about the CRUD operations. These are:

C: Create – When user ‘Save’ any new transaction, ‘Create’ operation is performed.
R: Retrieve – When user ‘Search’ or ‘View’ any saved transaction, ‘Retrieve’ operation is performed.

U: Update – when user ‘Edit’ or ‘Modify’ an existing record, the ‘Update’ operation is performed.
D: Delete – when user ‘Remove’ any record from the system, ‘Delete’ operation is performed.

It does not matter at all, which DB is used and how the operation is performed. End users performed lot of DB operations. But the end users are not concerned about any join or sub-query, trigger or stored-procedure, query or function was used to do what they wanted.

 

What we need to test in database testing:

We need to ensure some DB features/functions  for Database testing like Data Mapping, ACID test, Data Integrity, Implementation of Business role etc.

Ensure data mapping

We need to make sure that the mapping between different forms/screens of the application and the relations of its DB is not only accurate but is also according to design documents. We need to verify that respective tables and records are updated when user clicks ‘Save’, ‘Update’, ‘Search’ or ‘Delete’ from GUI of the application.

 

Ensure the ACID Properties of Transactions

ACID properties of DB Transactions refer to the ‘Atomicity’, ‘Consistency’, ‘Isolation’ and ‘Durability’. Proper testing of these four properties must be done during the DB testing activity. This area demands more rigorous, thorough and keen testing when the database is distributed.

Ensure Data Integrity

We need to consider that different modules of application use the same data in different ways and perform all the CRUD operations on the data. In that case, we need to make it sure that the latest state of data is reflected everywhere. System must show the updated and most recent values or the status of such shared data on all the forms and screens. This is called the Data Integrity.

Ensure Accuracy of implemented Business Rules:

Databases have been evolved into extremely powerful tools that provide sufficient support to the developers in order to implement the business logic at DB level. Some simple examples of powerful features of DBs are referential integrity, relational constrains, triggers and stored procedures. So, using these and many other features offered by DBs, developers implement the business logic on DB level. Test Engineers must ensure that the implemented business logic is correct and works accurately. They may execute the stored procedures, views to compare the result with real business.

Above points describe the four most important ‘What To’ of database testing. Now, I will briefly describe about ‘How To’ of DB Testing. But, first of all I like to strongly say that DB Testing is a business critical task, and it should never be assigned to a fresh or inexperienced resource without proper training.

 

How to Test Database:

We can perform the Database testing by the following below listed ways as a Software Test Engineer:

Create own Queries

First of all Test Engineer should have very good knowledge of SQL and specially DML statements in order to test the DB properly and accurately. Secondly, the tester should acquire good understanding of internal DB structure of application. If these two pre-requisites are fulfilled, then the Test Engineers are ready to test DB with complete confidence.  

We will perform any CRUD operation from the UI of application and we will verify the result using SQL query. This is the best and robust way of DB testing especially for applications with small to medium level of complexity. Yet, the two pre-requisites described are necessary. Otherwise, this way of DB testing cannot be adopted by the tester.

Moreover, if the application is very complex then it may be hard for the tester to write all of the needed SQL queries himself or herself. However, for some complex queries, tester may get help from the developer (or DB Specialist) too. I always recommend this method for the testers because it does not only give them the confidence on the testing they have performed but, also enhance their SQL skill.

Observe data table by table

The Test Engineers can verify the CRUD operation result by viewing the tables (relations) of DB, if they are not good at SQL. This way may be annoying and bulky (especially when the DB and tables have large amount of data).

Similarly, this way of DB testing may be extremely difficult for tester if the data to be verified belongs to multiple tables. This way of DB testing also requires at least good knowledge of Table structure of the application database.

Get query from developer/DB Specialist

Test engineer may collect the query from the developer or DB specialist.  This is the simplest way for the tester to test the DB. Perform any CRUD operation from GUI and verify its impacts by executing the respective SQL query obtained from the developer.

Tester will get knowledge about SQL by using this method. But, its drawback is havoc. What if the query given by the developer is semantically wrong or does not fulfill the user’s requirement correctly? In this situation, the client will report the issue and will demand its fix as the best case. While, the worst case is that client may refuse to accept the application.

Database is the core and critical part of almost every software application. So DB testing of an application demands keen attention, good SQL skills, proper knowledge of DB structure of application and proper training.

Penetration Testing: Some basic things that we need to perform

Now a day we are very concern about the security of our systems. So, we need to perform Penetration Testing for our systems. Penetration testing is also known as Pen Test. Penetration testing is the process to identify security vulnerabilities in an application by evaluating the system or network with various malicious techniques. Purpose of this test is to secure important data from outsiders like hackers who can have unauthorized access to system. Once vulnerability is identified it is used to exploit system in order to gain access to sensitive information.

Penetration is not the functional testing. In Pentest our goal is to find security holes in the system. Below are some generic test cases and not necessarily applicable for all applications.

• Check if web application is able to identify spam attacks on contact forms used in the website.
• Proxy server – Check if network traffic is monitored by proxy appliances. Proxy server makes it difficult for hackers to get internal details of the network thus protecting the system from external attacks.
• Spam email filters – Verify if incoming and outgoing email traffic is filtered and unsolicited emails are blocked. Many email clients come with in-build spam filters which needs to be configured as per your needs. These configuration rules can be applied on email headers, subject or body.
• Firewall – Make sure entire network or computers are protected with Firewall. Firewall can be a software or hardware to block unauthorized access to system. Firewall can prevent sending data outside the network without your permission.
• Try to exploit all servers, desktop systems, printers and network devices.
• Verify that all usernames and passwords are encrypted and transferred over secured connection like https.
• Verify information stored in website cookies. It should not be in readable format.
• Verify previously found vulnerabilities to check if the fix is working.
• Verify if there is no open port in network.
• Verify all telephone devices.
• Verify WIFI network security.
• Verify all HTTP methods. PUT and Delete methods should not be enabled on web server.
• Password should be at least 8 characters long containing at least one number and one special character.
• Username should not be like “admin” or “administrator”.
• Application login page should be locked upon few unsuccessful login attempts.
• Error messages should be generic and should not mention specific error details like “Invalid username” or “Invalid password”.
• Verify if special characters, html tags and scripts are handled properly as an input value.
• Internal system details should not be revealed in any of the error or alert messages.
• Custom error messages should be displayed to end user in case of web page crash.
• Verify use of registry entries. Sensitive information should not be kept in registry.
• All files must be scanned before uploading to server.
• Sensitive data should not be passed in urls while communicating with different internal modules of the web application.
• There should not be any hard coded username or password in the system.
• Verify all input fields with long input string with and without spaces.
• Verify if reset password functionality is secure.
• Verify application for SQL Injection.
• Verify application for Cross Site Scripting.
• Important input validations should be done at server side instead of JavaScript checks at client side.
• Critical resources in the system should be available to authorized persons and services only.
• All access logs should be maintained with proper access permissions.
• Verify user session ends upon log off.
• Verify that directory browsing is disabled on server.
• Verify that all applications and database versions are up to date.
• Verify url manipulation to check if web application is not showing any unwanted information.
• Verify memory leak and buffer overflow.
• Verify if incoming network traffic is scanned to find Trojan attacks.
• Verify if system is safe from Brute Force Attacks – a trial and error method to find sensitive information like passwords.
• Verify if system or network is secured from DoS (denial-of-service) attacks. Hacker can target network or single computer with continuous requests due to which resources on target system gets overloaded resulting in denial of service for legit requests.

These are just the basic test scenarios to get started with Pentest. There are hundreds of advanced penetration methods which can be done either manually or with the help of automation tools.

Finally as a penetration tester you should collect and log all vulnerabilities in the system. Don’t ignore any scenario considering that it won’t be executed by end users.

Way of Measuring Software Testing

Every step of Software Testing Process needs to be measured so as to guarantee a quality product to the customer. At the same time, measurement needs to be easy to understand and implement. Software testing process needs to follow the way of measurement:

·Size of Software: Software size means the amount of functionality of an application. The complete project estimation for testing depends on determining the size of the software. Many methods are available to compute the size software.

Widely used methods are:

o Function Point Analysis

o Use Case Estimation Methodology

· Requirement review: Software Requirement Specifications (SRS) should be obtained from the client before starting measurement of software testing process. This SRS should be:

o Complete: A complete requirements specification must precisely define all the real world situations that will be encountered and the capability's responses to them. It must not include situations that will not be encountered or unnecessary capability features.

o Consistent: A consistent specification is one where there is no conflict between individual requirement statements that define the behavior of essential capabilities and specified behavioral properties and constraints do not have an adverse impact on that behavior.

o Correct: For a requirements specification to be correct it must accurately and precisely identify the individual conditions and limitations of all situations that the desired capability will encounter and it must also define the capability's proper response to those situations.

o Structured: Related requirements should be grouped together and the document should be logically structured.

o Ranked: Requirement document should be ranked by the importance of the requirement.

o Testable: In order for a requirement specification to be testable it must be stated in such manners that pass/fail or quantitative assessment criteria can be derived from the specification itself and/or referenced information.

o Traceable: Each requirement stated within the SRS document must be uniquely identified to achieve trace ability. Uniqueness is facilitated by the use of a consistent and logical scheme for assigning identification to each specification statement within the requirements document.

o Unambiguous: A statement of a requirement is unambiguous if it can only be interpreted one way. This perhaps, is the most difficult attribute to achieve using natural language. The use of weak phrases or poor sentence structure will open the specification statement to misunderstandings.

o Validate: To validate a requirements specification all the project participants, managers, engineers and customer representatives, must be able to understand, analyze and accept or approve it.

o Verification: Requirements specification requires review and analysis by technical and operational experts in the domain addressed by the requirements. In order to be verifiable requirement specifications at one Practical Measurements for Software Testing level of abstraction must be consistent with those at another level of abstraction. Review efficiency can then be computed.

Review efficiency is a metric that provides an insight on quality of review and testing conducted.

Review efficiency = 100 * Total number of defects found by reviews / Total number of project defects
High values indicate an effective review process is implemented and defects are detected as soon as they are introduced.

· Effectiveness of testing requirements: Measuring the effectiveness of testing requirements involves;

o Specification of requirements & maintenance of Requirement Trace-ability Matrix (RTM): Specification of requirements must include;

· SRS Objective

· SRS Purpose

· Interfaces

· Functional Capabilities

· Performance Levels

· Data Structures/Elements Safety

· Reliability

· Security/Privacy

· Quality

· Constraints & limitations

Once the requirements have been specified and reviewed the next step is to update the RTM. The RTM is an extremely important document. In its simplest form, it provides a way to determine if all requirements are tested. However, the RTM can do much more. For example,

In this example, the RTM is used as a test planning tool to help determine how many tests are required, what types of tests are required, whether tests can be automated or manual, and if any existing tests can be re-used. Using the RTM in this way helps ensure that the resulting tests are most effective.

o Measuring the mapping of test cases to requirements: While we map the test cases to corresponding requirements, we must first determine the following:

· Number of requirements it tests

· Priority of requirement it tests

· Effort for its execution versus coverage of requirements

Here, the number of requirements it tests and the priority of the requirements can be obtained from the Software requirement specifications (SRS). We should see that every requirement has a set of test cases mapped to it.