Test Automation Services plays a vital role in establishing value as a testing partner and reducing testing costs in the long term from a customer perspective.
Several solutions in the industry are available today and the customer has a variety of choices that he can pick and choose from.
In fact there are some solutions that even offer robotic test execution. Irrespective one cannot deny the fact that Automation is the need of the hour and is a high impact lever for most customers.
The best of test automation is bought out in scenarios where the environment is Regression intensive.
By that, I mean if there are several things to do done in a repetitive fashion it lends itself better for an effective Return on Investment.
Dwelling a little deeper, in a large transformation engagement that I managed recently there were tasks that had to be done on a daily, weekly and monthly basis. The daily tasks were things to do with setting up the environment and test bed, making sure that the interrelated systems were all functioning etc.
These type of tasks were indeed repetitive in nature and a lot of human effort was required to get this going on a daily basis.
For instance someone had to burn the mid-night oil and make sure that a test environment was ready before the dawn of the day in another part of the world.
All this changed when we were able to established something called a Universal Automation Suite which contained these test automation scripts that would be automatically triggered day in and day out, notifying exceptions if any.
Only then would a human intervention be required. In addition this suite had all the test cases from an application stand point that would be executed over and over again.
Oh well, it took a hell a lot of time convincing the customer that this type of automation solution was the way around on the long run.
Rightly so, because most customer want ROI the next movement so that they can show it as part of their balance sheets and executive management will be happy seeing the numbers.
There is a general thumb rule that test automation leads to high returns only if an automated test case is executed at least three times.
The costs involved with automation is relatively high at the beginning as design of scripts plays a predominant part in the whole process and requires a fair amount of technical depth.
However this cost gets evened out during execution, because the process is simple and repetitive.
Selling test automation and more importantly convincing customers on effectiveness has always been a challenge and will continue to do so.
Bundling automation solutions is the key in creating a winning proposition. One novel way we did get to was to take a “Skin in the game” approach where the customer really never pays anything till the time he sees results.
As an example if I as a testing partner spend $’s in the design of a hundred test cases, I do not get paid unless I show that I have executed the test cases at least three times.
This is really a win-win situation for both the stakeholders. One key soft aspect that comes into play is something called “Transparency” in the whole process.
The claims on automation success can be debated by both the stakeholders without concrete benefit.
Therefore the need for solutions to integrate with test management tools is critical. When the actual and factual data/metrics is made transparent, most times customers do not have a “credibility” angle in commercial discussions.
Two key concepts that are gaining ground today in the automation world are around a) Mass Certification b) Test Data Management.
Mass certification is essentially a concept if proven right will saves tons on money and effort.
To break this concept a bit, lets us take a financial software application or product. Most times the end output after all business processing will result in a GL type of entity.
The only verification to find out if the software application/product is performing to its expected output is to compare the GL of a pre modified software vs after the changes have been implemented in it. Period. Oh wait a minute!
This is not as easy as it seems and requires hell a lot of things to be done before such a simple conclusion can be made. These challenges and constraints is what is making Mass certification more conceptual than objective.
I am sure that a lot of organizations are already implementing such things, though may not be in a large scale.
The second aspect is all about Test data management. For most of us it is not uncommon to mark test cases as “Not executed” for want of data.
In the middle of a business scenario we get stuck for want of data. Several discussions with business stakeholders and infrastructure personal at the customer organization only leads to elongated test life cycle and in turn hurt burns.
Quickly extracting the test data from a large database that suits the conditions that are to be tested is a huge “Differentiator”, in the full test life cycle.
This is where ETL offers great workarounds. A fantastic UI built on a Cognos like platform could extract the data real quickly.
What is interesting is that it could be done by a tester himself, rather than going to a business user or a data center professional.
In conclusion, I strongly believe that the following factors are essential for a successful automation program.
1) A Commercial model that results in a win-win situation
2) Techniques that give customers a quicker ROI like Mass Certification
3) Integrating with Test Management tools that create the much required “Transparency”
4) Data extraction tools bundled as part of Automation solutions that will help reduce test execution life time.
Automation is too small a topic to be ignored in any kind of test engagement. Happy Automation!
