Test Automation vs. Test Execution
Digitalization of each and any aspect of our lives brings the urge for companies to deliver their software products and services faster to the end customers, without sacrificing their quality. Without any doubt, automating testing processes and quality processes plays a key role in achieving this and more and more companies nowadays pay higher attention to test automation and replacing manual testing with automated test cases.
Going this way, many organizations are focused on getting a high level of test automation coverage, driven by targets and numbers. Sadly, these automated test cases are barely executed after their creation and when the next release starts knocking on the door, the team realizes that these test cases are outdated, merely working and need a serious amount of time to be brought in a running state. Meanwhile, to fulfill the coverage targets, the test specialists must create more new test automation scripts.
Each automation test case that is not part of any execution and investigation cycle, gets you to the maintenance trap
This situation surely brings frustration. On the one hand the management spends huge slice of the budget on test automation to speed up delivery process. On the other hand, the testing group should fight with constantly growing maintenance work (bringing not working/outdated testing into working state) while reaching automation targets.
Purpose of testing in mind
A higher test automation rate does not automatically mean ‘good’ or ‘right’ testing. It is of course one of the important quality related KPIs, however it is most important how and for what purposes a given company is using its automated test cases.
To understand the main purpose of the testing we need to underline once again its role in the whole Quality Assurance process.
Even though testing is one the main and most important activities in QA, it does not directly impact the quality of the product/service. The main purpose of testing is to provide information about current state of the AUT (Application Under Test) and answer to the most important question of the business: “Can I go on production with this version?”
And answer to this question is possible only through frequent executions of the existing automation scrips.
Why do we need to focus on test execution?
Less is more
Switching the focus to test execution and on covering the business risk first bring direct impact on improving the delivery process. Less and working test cases which are executed on a daily basis bring more benefits than high number of automated test cases which are not working.
Deep knowledge over the stability of the system
Frequent investigations of failed test cases give you good knowledge not only of the system but on the automation test cases as well. In this way we can fully relay on our testing and most importantly provide the team and stakeholder with accurate answer if we can proceed further or not in the release chain.
Detect defect early
Recurrent test executions detect regression errors earlier where the risk of them is not that high. The team can fix them faster as the problematic part of the code can be easy isolated.
Keeping the system healthy
Let’s say it laud. Executions of automation test cases detect not only problems with functionality of the system, but major problems in integrated systems, performance or with infrastructure as well. With each investigation we bring stability to the system and keep it in a healthy state.
5 Tips to switch the focus easily to test execution
#1
Set up execution environment from the very beginning
Even if there are 5 test cases, make sure that they are scheduled and executed automatically. With every new cycle, new test cases can be brought to the execution environment.
#2
Execute your automation test scripts/cases daily
The main purpose of creating automation test cases is to replace the manual work of anyone involved in testing. Why don’t you execute them during the night, when the environment is free and have results next day, in the morning?
#3
Investigation of test execution results first
Do not start any new tasks before you investigate the outcome form your nightly executions. There will be red test cases and your job as a tester is to communicate with your team and stakeholder the findings. Sometimes there are critical errors which need to be removed immediately.
#4
Do not panic when there are red test cases after execution
If you work in an agile environment and your test cases are running constantly green, then there is something wrong with your test cases. Having green test executions is a great thing while you are one stage before release, but on earlier stage, automation test cases should detect every modification in the software, no matter if it is caused by desired change or introduced by error. In any case, any change is detected, and you need to take proper actions, either to adapt the automation test scrip accordingly or raise a defect.
#5
Plan maintenance work
There are automation scripts which runs unstable, so-called false positives, and they should be taken out of test executions and improved. By focusing on executions, you have the understanding which scripts cause problem and request time to stabilize them, when the project is not in a risk phase of its live cycle and their number is not that high.
If you need to adjust your testing and QA strategy in a way that it fulfils the new requirements of the industry but you are straggling from where to start, do not hesitate to contact us and schedule a free consultation session. Our testOps professionals will be glad to guide you through the process.
