Some misconceptions about agile testing

Agile development is become popular to organizations day by day. Testing persons needs to perform ‘Agile Testing’ for agile development. There are lots of misconceptions about agile testing. Some very common myths are discussed below:

Myth1: Agile Testing is ad hoc

Ad hoc testing is performed without planning and documentation. Agile involves with planning sprints and resources forward of time. The test cycles are sync with development cycles; that are an inherent part of the planning.  The agile test plans are based on the user stories of a sprint. The stories may be included or removed from a sprint. Agile testing is centered on the developed stories; so, it is not ad hoc.

Myth2: Testing in Agile is undocumented, quick and dirty

Documentation is an important part of agile project. Agile insists on rapid communication. The only difference with other mythologies is that agile gives more important to face-to-face interaction than extreme dependence on written communication. Agile maintains tools for communication.  

The agile teams are usually provided with guidelines and requirements on the level of documentation and coding comments that they need to incorporate.

Myth3: Agile Testing does not have defined strategies

Testing cycle is synchronized with development, and it acts as quality gateways for the sprints in agile project. Multiple approaches are followed to quickly test the stories of a sprint. Also, agile testing helps in continuous monitoring of the application after release; and ensuring a complete feedback loop for the next sprints.

Myth4: Agile testing is tool oriented

Agile is a person oriented methodology. Agile gives more importance to person over process and tools. Efficient development and testing practices are more important than just being able to use tools. Test teams are engaged with daily sprint meetings from the start of user story identification. So, it helps to identify issues on early in the sprint.

Myth5: Agile Testing is best done by developers

Testing is inspired through the agile methodology. It is done by both developers and testers. It is a collaborative work approach between developers and testers. Testing teams are seamlessly integrated with the development life-cycle. The role of a tester in agile becomes more proactive.  

Testers seamlessly collaborate with the developers to meet their goal of delivering a bug free product by finding the bugs, fixing those, and verifying those before a sprint release.

Myth6: Agile Testing should all be at same geographical location

It is definitely good to have the whole team in one geographical location. Teams are assembled from different geographical locations with the help of technology in a global company. Meetings can be held involving people all around the world at the click of a button. There are some tools available for collaboration and file sharing.

Testing teams should be ready to accommodate different time zones and leverage technology to adopt efficient meeting practices. With a little effort, flexibility, adaptability and the proper mind-set, distributed teams can just be as agile as the co-located teams.

So, try to avoid the above listed misconceptions and start Agile from today.

What & How to test a Database – The Basics

Today I’ll discuss about some basic techniques of the database testing. A Test Engineer needs to test DB on the testing phase of software. Database is the most important part of any software. It is impossible for us to develop software without Database. It does not matter at all whether it is web or desktop, client server or peer to peer, enterprise or individual business, database is working at back-end.

Currently, several database tools are available in the market like MS-Access, MS SQL Server, Oracle, MySQL, PostgreSQL, DB2 etc.  All of these vary in cost, robustness, features and security. Each of these DBs possesses its own benefits and drawbacks. One thing is certain; a business application must be built using one of these or other DB Tools.

Before I start the What & How topic, I would like to share some common characteristics of Database. The end user mainly operates the CRUD (Create, Retrieve, Update, and Delete) operations when the application is under execution. Every Test Engineer needs to know at least the basic about the CRUD operations. These are:

C: Create – When user ‘Save’ any new transaction, ‘Create’ operation is performed.
R: Retrieve – When user ‘Search’ or ‘View’ any saved transaction, ‘Retrieve’ operation is performed.

U: Update – when user ‘Edit’ or ‘Modify’ an existing record, the ‘Update’ operation is performed.
D: Delete – when user ‘Remove’ any record from the system, ‘Delete’ operation is performed.

It does not matter at all, which DB is used and how the operation is performed. End users performed lot of DB operations. But the end users are not concerned about any join or sub-query, trigger or stored-procedure, query or function was used to do what they wanted.

 

What we need to test in database testing:

We need to ensure some DB features/functions  for Database testing like Data Mapping, ACID test, Data Integrity, Implementation of Business role etc.

Ensure data mapping

We need to make sure that the mapping between different forms/screens of the application and the relations of its DB is not only accurate but is also according to design documents. We need to verify that respective tables and records are updated when user clicks ‘Save’, ‘Update’, ‘Search’ or ‘Delete’ from GUI of the application.

 

Ensure the ACID Properties of Transactions

ACID properties of DB Transactions refer to the ‘Atomicity’, ‘Consistency’, ‘Isolation’ and ‘Durability’. Proper testing of these four properties must be done during the DB testing activity. This area demands more rigorous, thorough and keen testing when the database is distributed.

Ensure Data Integrity

We need to consider that different modules of application use the same data in different ways and perform all the CRUD operations on the data. In that case, we need to make it sure that the latest state of data is reflected everywhere. System must show the updated and most recent values or the status of such shared data on all the forms and screens. This is called the Data Integrity.

Ensure Accuracy of implemented Business Rules:

Databases have been evolved into extremely powerful tools that provide sufficient support to the developers in order to implement the business logic at DB level. Some simple examples of powerful features of DBs are referential integrity, relational constrains, triggers and stored procedures. So, using these and many other features offered by DBs, developers implement the business logic on DB level. Test Engineers must ensure that the implemented business logic is correct and works accurately. They may execute the stored procedures, views to compare the result with real business.

Above points describe the four most important ‘What To’ of database testing. Now, I will briefly describe about ‘How To’ of DB Testing. But, first of all I like to strongly say that DB Testing is a business critical task, and it should never be assigned to a fresh or inexperienced resource without proper training.

 

How to Test Database:

We can perform the Database testing by the following below listed ways as a Software Test Engineer:

Create own Queries

First of all Test Engineer should have very good knowledge of SQL and specially DML statements in order to test the DB properly and accurately. Secondly, the tester should acquire good understanding of internal DB structure of application. If these two pre-requisites are fulfilled, then the Test Engineers are ready to test DB with complete confidence.  

We will perform any CRUD operation from the UI of application and we will verify the result using SQL query. This is the best and robust way of DB testing especially for applications with small to medium level of complexity. Yet, the two pre-requisites described are necessary. Otherwise, this way of DB testing cannot be adopted by the tester.

Moreover, if the application is very complex then it may be hard for the tester to write all of the needed SQL queries himself or herself. However, for some complex queries, tester may get help from the developer (or DB Specialist) too. I always recommend this method for the testers because it does not only give them the confidence on the testing they have performed but, also enhance their SQL skill.

Observe data table by table

The Test Engineers can verify the CRUD operation result by viewing the tables (relations) of DB, if they are not good at SQL. This way may be annoying and bulky (especially when the DB and tables have large amount of data).

Similarly, this way of DB testing may be extremely difficult for tester if the data to be verified belongs to multiple tables. This way of DB testing also requires at least good knowledge of Table structure of the application database.

Get query from developer/DB Specialist

Test engineer may collect the query from the developer or DB specialist.  This is the simplest way for the tester to test the DB. Perform any CRUD operation from GUI and verify its impacts by executing the respective SQL query obtained from the developer.

Tester will get knowledge about SQL by using this method. But, its drawback is havoc. What if the query given by the developer is semantically wrong or does not fulfill the user’s requirement correctly? In this situation, the client will report the issue and will demand its fix as the best case. While, the worst case is that client may refuse to accept the application.

Database is the core and critical part of almost every software application. So DB testing of an application demands keen attention, good SQL skills, proper knowledge of DB structure of application and proper training.

Penetration Testing: Some basic things that we need to perform

Now a day we are very concern about the security of our systems. So, we need to perform Penetration Testing for our systems. Penetration testing is also known as Pen Test. Penetration testing is the process to identify security vulnerabilities in an application by evaluating the system or network with various malicious techniques. Purpose of this test is to secure important data from outsiders like hackers who can have unauthorized access to system. Once vulnerability is identified it is used to exploit system in order to gain access to sensitive information.

Penetration is not the functional testing. In Pentest our goal is to find security holes in the system. Below are some generic test cases and not necessarily applicable for all applications.

• Check if web application is able to identify spam attacks on contact forms used in the website.
• Proxy server – Check if network traffic is monitored by proxy appliances. Proxy server makes it difficult for hackers to get internal details of the network thus protecting the system from external attacks.
• Spam email filters – Verify if incoming and outgoing email traffic is filtered and unsolicited emails are blocked. Many email clients come with in-build spam filters which needs to be configured as per your needs. These configuration rules can be applied on email headers, subject or body.
• Firewall – Make sure entire network or computers are protected with Firewall. Firewall can be a software or hardware to block unauthorized access to system. Firewall can prevent sending data outside the network without your permission.
• Try to exploit all servers, desktop systems, printers and network devices.
• Verify that all usernames and passwords are encrypted and transferred over secured connection like https.
• Verify information stored in website cookies. It should not be in readable format.
• Verify previously found vulnerabilities to check if the fix is working.
• Verify if there is no open port in network.
• Verify all telephone devices.
• Verify WIFI network security.
• Verify all HTTP methods. PUT and Delete methods should not be enabled on web server.
• Password should be at least 8 characters long containing at least one number and one special character.
• Username should not be like “admin” or “administrator”.
• Application login page should be locked upon few unsuccessful login attempts.
• Error messages should be generic and should not mention specific error details like “Invalid username” or “Invalid password”.
• Verify if special characters, html tags and scripts are handled properly as an input value.
• Internal system details should not be revealed in any of the error or alert messages.
• Custom error messages should be displayed to end user in case of web page crash.
• Verify use of registry entries. Sensitive information should not be kept in registry.
• All files must be scanned before uploading to server.
• Sensitive data should not be passed in urls while communicating with different internal modules of the web application.
• There should not be any hard coded username or password in the system.
• Verify all input fields with long input string with and without spaces.
• Verify if reset password functionality is secure.
• Verify application for SQL Injection.
• Verify application for Cross Site Scripting.
• Important input validations should be done at server side instead of JavaScript checks at client side.
• Critical resources in the system should be available to authorized persons and services only.
• All access logs should be maintained with proper access permissions.
• Verify user session ends upon log off.
• Verify that directory browsing is disabled on server.
• Verify that all applications and database versions are up to date.
• Verify url manipulation to check if web application is not showing any unwanted information.
• Verify memory leak and buffer overflow.
• Verify if incoming network traffic is scanned to find Trojan attacks.
• Verify if system is safe from Brute Force Attacks – a trial and error method to find sensitive information like passwords.
• Verify if system or network is secured from DoS (denial-of-service) attacks. Hacker can target network or single computer with continuous requests due to which resources on target system gets overloaded resulting in denial of service for legit requests.

These are just the basic test scenarios to get started with Pentest. There are hundreds of advanced penetration methods which can be done either manually or with the help of automation tools.

Finally as a penetration tester you should collect and log all vulnerabilities in the system. Don’t ignore any scenario considering that it won’t be executed by end users.

Reporting a bug is nothing but a skill

“Reporting a bug is nothing but a skill” – it is my belief. If your bug report is effective, chances are higher that it will get fixed faster. So fixing a bug depends on how effectively you report it. If test engineer is not reporting bug correctly, programmer will most likely reject this bug stating as irreproducible. This can hurt test engineers moral and sometime ego also. But I suggest, as a Software Test Engineer do not keep any type of ego like “I have reported bug correctly”, “I can reproduce it”, “Why he/she has rejected the bug?”, “It’s not my fault” etc.

What are the qualities of a good software bug report?

Anyone can write a bug report. But not everyone can write an effective bug report. You should be able to distinguish between average bug report and a good bug report. How to distinguish a good or bad bug report? It’s simple; I think you may apply following characteristics and techniques to report a bug.

• Having clearly specified bug number: Always assign a unique number to each bug report. This will help to identify the bug record. If you are using any automated bug-reporting tool then this unique number will be generated automatically each time you report the bug. Note the number and brief description of each bug you reported.
  
• Steps to Reproduce: If your bug is not reproducible it will never get fixed. You should clearly mention the steps to reproduce the bug. Do not assume or skip any reproducing step. Step by step described bug problem is easy to reproduce and fix.
• Be Specific: Do not write an essay about the problem. Be Specific and to the point. Try to summarize the problem in minimum words yet in effective way. Do not combine multiple problems even they seem to be similar. Write different reports for each problem.

How to Report a Bug?

I like to describe a simple but effective bug report format/template in below. It may vary on the bug report tool you are using. If you are writing bug report manually then some fields need to specifically mention like Bug number which should be assigned manually. Your bug report will contain:

· Reporter Description: Your name and email address.

· Product: In which product you found this bug.

· Version: The product version where you found this bug.

· Component: These are the major sub modules of the product. Sometimes it may be the form name.

· Platform: Mention the hardware platform where you found this bug. The various platforms like ‘PC’, ‘MAC’, ‘iPod’ etc.

· OS & Version: Mention the operating systems and the version where you found the bug. Operating systems like Windows, Linux, UNIX, Sun OS, Mac OS. Mention the different OS versions like Windows NT, Windows 7, Windows XP etc.

· Priority: When bug should be fixed? Priority is generally set from P1 to P5. Or “Low” to “Immediate’.

· Severity: Severity describes the impact of the bug. Types of Severity may:

o Blocker: No further testing work can be done.

o Critical: Application crash, Loss of data.

o Major: Major loss of function.

o Minor: minor loss of function.

o Trivial: Some UI enhancements.

· Status: When you are logging the bug in any bug tracking system then by default the bug status is ‘New’. Later on bug goes through various stages like Assigned, Resolved, Closed etc.

· Assign To: If you know which developer is responsible for that particular module in which bug occurred, then you can specify credential of that developer. Else keep it blank this will assign bug to module owner or manager will assign the bug to developer. Possibly add the manager email address in CC list.

· URL: Mention the page url on which bug occurred.

· Summary: A brief summary of the bug mostly in 60 or below words. Make sure your summary is reflecting what the problem is and where it is.

· Description: A detailed description of bug. Use following fields for description field:

o Reproduce steps: Clearly mention the steps to reproduce the bug.

o Expected result: How application should behave on above mentioned steps.

o Actual result: Write the actual result that will produce on running above steps.

You may also concern about the following to write a good bug report:

1) Report the problem immediately: If you find any bug while testing, do not wait to write detail bug report later. Instead write the bug report immediately. This will ensure a good and reproducible bug report. If you decide to write the bug report later on then chances are high to miss the important steps in your report.

2) Reproduce the bug three times before writing bug report: Your bug should be reproducible. Make sure your steps are robust enough to reproduce the bug without any ambiguity. If your bug is not reproducible every time you can still file a bug mentioning the periodic nature of the bug.

3) Test the same bug occurrence on other similar module: Sometimes developer use same code for different similar modules. So chances are high that bug in one module can occur in other similar modules as well. Even you can try to find more severe version of the bug you found.

4) Write a good bug summary: Bug summary will help developers to quickly analyze the bug nature. Poor quality report will unnecessarily increase the development and testing time. Communicate well through your bug report summary. Keep in mind that bug summary is used as a reference to search the bug in bug inventory.

5) Read bug report before hitting Submit button: Read all sentences, wording, steps used in bug report. See if any sentence is creating ambiguity that can lead to misinterpretation. Misleading words or sentences should be avoided in order to have a clear bug report. You musr check the spell.

6) Do not use Abusive language: It’s nice that you did a good work and found a bug but do not use this credit for criticizing developer or to attack any individual.

No doubt that your bug report should be a high quality document. Focus on writing good bug reports, spend some time on this task because this is main communication point between tester, developer and manager. Mangers should make aware to their team that writing a good bug report is primary responsibility of any tester. Your efforts towards writing good bug report will not only save company resources but also create a good relationship between you and developers.

So, for better productivity write a better bug report.