In agile development test automation has significant role. Test automation includes testing techniques like automated acceptance and regression testing and unit testing.
Without automation testing team gets stuck in long and boring manual testing. But automation testing services needs to be done accurately, with poorly done automation you will get into these two situations where developers will shout at you for continuous rejection of builds and your project manager considers that you are providing broken code.
Commonly during the early stages of automation test scripting there are situations where some tests pass randomly this is caused by set of factors like different order of page loading one of the most common irritation is when tests works fine in local environment but fails on build agent this causes debugging very time consuming process. Due to this there is a great requirement of improving test automation quality and making the tests more robust.
This goal of this blog is to provide a set of methods to utilize in situations when you notice that your tests running but failing sometimes then run occasionally and failing for longer again.
-
Avoid usage of record and playback scripts for automation
Using record and playback tools like selenium IDE we get the unmaintainable and fragile scripts and for Agile development one cannot spend such a long time to fix these fragile scripts.
-
Automation needs a collaborative approach
it is responsibility of whole team, don’t allocate automation task to one person. Weave you goals of automation in scrum team.
-
Staying on the top of the failures
The causes of test failures are test instability, environmental problems and issues in code. Tests which never come into failures are always ignored and tests which mostly fail are always omitted.
-
Transparent Communication between team members
Build outcomes are communicated by information radiators between team mates. Conventions used must be easily understood by team mates like failure in “Red”, success in “Green” and make usage of trend graphs for more understanding.
-
Adding Tags with tests
Test’s run frequency is specified by tags so multiple tags must be attached to check run frequency of tests.
-
Tests in isolation
Small tests must be written for individual components before and end to end testing so as to isolate the failures. Writing large tests for fully integrated system causes whole test suit to fail.
-
Make your custom Exception
Exception names are enough to give understanding of failures so you must extend exception class in your own class to make your custom exceptions which will give more information to you and other persons while debugging your tests.
Always Make Sure To Conduct a Pre-Test Checkups
There are cases where flaky tests are the problem but unready environments are the bigger problem than that. You must create a robust set of tests which you can treat as a pre-flight check. Before you run any integration tests, you must run this suite to understand the compatibility of series of tests integration. These tests are comparatively very quick as they only check the HTTP statuses and integration tests.
The Debugging & Retries
Now, imagine if a test fails for three consecutive times. In that condition, you must automate the retry. It might increase the run times of the test; however, it will act as a quick fix until you take manual time to debug the cause of failure properly. You must perform this method until a degree of test stability is attained.
