I’ve written the blog for find out the current stage of a QA/Test Engineer in Software Engineering world. Here I’ve listed the criterion for Level10, Level11, and Level12. It is mainly based on your working experience, reading familiarize, and the professional certification. For achieving a level, you need to have some practical working experience, Professional certification, and also you have to read some books. Book reading is important due to you may apply the techniques to your job that you learn from the books.
Level 10
During this time, the engineer obtains introductory capability in all knowledge areas and competence in quality, test, and process.
Activity Type
Details
Work Experience
Act as a tester on at least one project
Act as a backup quality lead on at least one project
Write one or more test case specification documents on a project
Participate in the release process of a project
Perform personal planning and tracking on a project
Participate in a code review
Participate in the review of a quality plan, test plan, and test cases
Participate in an informal review
Participate in an inspection
Review a project's documentation including the project plans, schedules, work breakdown structures, designs, and code
Reading
The Art of Software Testing, Glenford Myers
Code Complete, Steve McConnell
Rapid Development, Steve McConnell
Software Project Survival Guide, Steve McConnell
UML Distilled, Martin Fowler et all
Software Measurement Guidebook", NASA Goddard Space Flight Center
Level 11
During this time, the engineer obtains competence in requirements, engineering management, and construction along with leadership in test.
Activity Type
Details
Work Experience
Act as the quality lead on at least one major project
Software Testing is one of the key practices in the Software Development Life Cycle that requires diversified skills. Because, developers find it difficult to find out the defects in their own code psychologically, the developers cannot test their code effectively. Hence, there arises the need for an Independent Testing Group.
Every Test Engineer need to have some skill sets. The most important seven (7) Skill sets based on me are:
1. Understanding Skills The first and foremost activity of Software Testing is to understand the requirements/ functionalities of the software to be tested. Formal Documents like Software Requirement Specifications, Software Functional Specifications, Use Case Specifications and other documents like Minutes of Meeting serves as the key references for planning and executing the testing tasks. The test engineer must have very good understanding skills to read and understand the information written in these documents. Many a times, it is possible to have many interpretations for the information presented in the documents. The test engineers must be able to identify the duplicate & ambiguous requirements.
2. Listening Skills Documents are not the only source of reference for the testing activities. The information required for the testing activities may be acquired through offline meetings, seminars, conferences, etc. The minutes of the meetings, conferences, seminars may or may not be recorded in a formal document. The Test Engineers must have very good active listening skills in order to collate and co-relate all of that information and refer them for the testing activities. While the requirements or functionalities of the software are discussed over a meeting, many a times, some part of the requirements are missed out. The Test Engineers should be able to identify and get them clarified before heading towards the subsequent testing phases. 3. Test Planning Skills All software requirements shall be testable. The software shall be designed in such a way that all software requirements shall be testable. The test plan shall be formulated in such a way that paves the way for validating all the software requirements. In the real time scenario, there could be many requirements that are not testable. The Test Engineer with his/her test planning skills should be able to find out a workaround to test those non-testable requirements. If there is no way to test them, that shall be communicated clearly to the appropriate authority. There could be many requirements that are very complex to test and the Test Engineer should be able to identify the best approach to test them.
4. Test Design Skills Software Testing Science preaches many techniques like Equivalence Class Partitioning, Boundary Value Analysis, Orthogonal Array and many more techniques for an effective test design. The Test Engineers shall be aware of all those techniques and apply them into their software test designing practice. The Test Engineer shall be aware of the how to write test cases – non-ambiguous, simple, straight to the point. The test case needs to contain Test Case Description, Test Steps and its corresponding expected results. 5. Test Execution Skills Test Execution is nothing but executing the steps that is specified in the test design documents. During the execution, the Test Engineers shall capture the actual results and compare against the expected results specified in the test design documents. If there are any deviations between the expected and actual results, the Test Engineers shall consider that as a defect. The Test Engineer shall analyze the cause of the defect and if it is found and confirmed in the application under test, the same shall be communicated to the developers and it shall get fixed. If the cause of the defect is found with the test case, the same shall be communicated to the test designers and the test cases shall be modified/ amended accordingly. If the Test Engineers are not confident about the application functionalities and the test design documents, they may not confidently come to a conclusion about the defect in case of any discrepancies. This will lead to defects being leaked to the next phase and the Test Engineers needs to avoid this scenario. The Test Engineers shall be confident about the application functionalities and in case of any ambiguity or clarifications; they need to get them sorted out before executing the tests or at least during the test execution.
6. Defect Reporting Skills Defect Reports are one of the critical deliverables from a Test Engineer. The defect reports are viewed by the development team, business analysts, project managers, technical managers and the quality assurance engineers along with the Test Engineers. Hence, the defect reports shall carry enough information about the defect. Steps to reproduce the defect, its expected result and actual result along with other information such as Severity, Priority, Assigned To (developer), Test Environment details are very critical for a defect report without which, the defect report is considered as incomplete. The Test Engineer shall be aware of the importance of the defect report and he/she shall create defect report in such a way that it is non-ambiguous. During the course of fixing the defect, the developers may come back to the testing team for more information regarding the defect and the Test Engineer shall provide the same without failing.
7. Test Automation Skills Test Automation is a wonderful phenomenon by which the testing cost is drastically reduced. The manual test cases upon automation can be executed by running the automated test scripts. This means that the manual effort to run those automated test cases is not necessary and hence the total test effort is reduced drastically. The Test Engineers shall be aware of the technique for adopting test automation into the current testing process. Identifying the test automation candidates is very critical for the success of the automation project. Automation candidates shall be identified in such a way that the testing cost towards manual test execution would reduce significantly. This involves lots of thoughts from the financial perspective as well. The Test Engineers shall understand the process of do’s & don’ts of automation to make the automation project successful.
The Test Engineers shall understand/ learn and be confident about the application functionalities. Test planning, designing, execution and defect reporting are the basic and essential skills that a Test Engineer shall possess and develop in his day-to-day career.
All Test Engineers come across this typical question as to "when to stop" testing. Fact is that testing can never be considered complete. We can never be able to prove scientifically that our software system is free from errors now.
Most of the time we are facing the following conditions:
Stop the testing when the committed testing deadlines are expiring.
Stop the testing when we are not able to detect any more errors even after execution of all the planned test Cases.
We can see that both the above statements do not carry any meaning and are contradictory since we can satisfy the first statement even by doing nothing while the second statement is equally meaningless since it can not ensure the quality of our test cases.
When to stop testing is difficult. Many modern software applications are so complex and run in such an Interdependent environment, that complete testing can never be done.
Most of the time we follow the following conditions:
Stop the Testing when deadlines like release deadlines or testing deadlines have reached.
Stop the Testing when the test cases have been completed with some prescribed pass percentage.
Stop the Testing when the testing budget comes to its end.
Stop the Testing when the code coverage and functionality requirements come to a desired level.
Stop the Testing when bug rate drops below a prescribed level.
Stop the Testing when the period of beta testing / alpha testing gets over.
Testing metrics can help the Test Engineer to take better and accurate decisions; like when to stop testing or when the application is ready for release.
The best way is to have a fixed number of test cases ready well before the beginning of test execution cycle. Subsequently measure the testing progress by recording the total number of test cases executed.
We can use the following metrics that are quite helpful in measuring the quality of the software product.
Percentage Completion:
(Number of executed test cases) / (Total number of test cases)
Percentage Test cases Passed:
(Number of passed test cases) / (Number of executed test cases)
Percentage Test cases Failed:
(Number of failed test cases) / (Number of executed test cases)
A test case is declared - Failed even when just one bug is found while executing it, otherwise it is considered as - Passed
There are some Scientific Methods to decide when to stop testing. These are:
Decision based upon Number of Pass/Fail test Cases:
Preparation of predefined number of test cases ready before test execution cycle.
Execution of all test cases in every testing cycle.
Stopping the testing process when all the test cases get passed.
Alternatively testing can be stopped when percentage of failure in the last testing cycle is observed to be extremely low.
Decision based upon Metrics:
Mean Time Between Failures (MTBF): by recording the average operational time before the system failure.
Coverage metrics: by recording the percentage of instructions executed during tests.
Defect density:by recording the defects related to size of software like "defects per 1000 lines of code" or the number of open bugs and their severity levels.
Finally how to decide Stop the testing?
If:
Coverage of the code is good
Mean time between failure is quite large
Defect density is very low
Number of high severity Open Bugs is very low.
After even doing all, Test Engineers are not pleased about their job. They think some testing is not done yet. There are some gap about their test case and the business. Fact is that Test engineer can never be considered test complete.
Good testers must not shy away from argument. It’s their job to report bad news—and this is not always taken well. Sometimes it’s hard to pinpoint the source of a bug. Is it a design bug, a coding bug, or maybe a documentation bug? Or perhaps the tester is just confused, and it’s not a real bug at all. Regardless, the tester’s job is to report the problem. Some testers may go overboard, thinking it’s their job to pass judgment on the developers or force them to do better work. This is not helpful. But good testers need to be the kind of people who are not afraid of other people’s reactions to bad news. On the other hand, developers often need to avoid the emotional intensity that makes concentration difficult. It is not easy concentrating on complicated technical material hour after hour. I saw developers on my project take much more time diagnosing problems than I would have. It is important to report problems as soon as they’ve been confirmed, and good testers will report a problem once it is reproducible and narrowed down. I saw developers, however, move right into debugging, identifying the faulty code and the build that introduced it. Part of this is attributable to the fact that they had the skills and interest in debugging the code. But it also seemed as if they were reluctant to report that they had found a problem until they knew exactly what they had found. They wanted to avoid a situation where people started speculating as to who or what may have a caused a problem. Testers need to demonstrate that they can continue to think and work in the face of conflict. I want to see that they will be tenacious, and that they have some faith in their own perspective. Some people have strong technical skills, but are very uncomfortable in these kinds of situations.
Few days ago, I was testing a web based software. In the mean time I was working with the user security system. I have created several users with different level permission. These were performing successfully. Then I’ve changed the user level and tested those. In that case these were performing well. After some time I was engaging with delete user. On that time I was log in by using Admin user. I’ve successfully deleted some inactive users, who have not made any activity or transaction on the software. Then I was trying to delete some users who have performed some task. Surprisingly these Users have deleted. Lastly a crack experience was waiting for me. I just click to delete for my currently using user that is Admin. I was thundered the ADMIN was deleted.
So, please every one try to do this; If you testing the Security Module.
