In today’s competitive business world, it is unavoidably important that the businesses have as much profit as possible out of what they have to sell. It’s like squeezing all the juice out of an orange to make the most amount of juice. But of course, this is expected as growth rate is important in the business world. The situation is not so different in software development businesses either.

But of course, software development is mostly time variant more than any other business types because other parameters affect the processes minimally, and the latency to assert a totally new product or a developed version of a product into the market takes too long a time. And this period of time is even more elongated with the allocated time for testing the products. However, to make sure a product is actually good, testing procedures are crucial to being followed. If it is considered unnecessary that the product is good, think again, because growth is only made with sales. Sales are easily increased with the quality of the product. Therefore, to get a steady growth, sales have to be made, and for that, products have to be good quality, and tests are important to have good quality.

Considering the need for steady growth and the need for good quality products in the market, testing needs to be done but needs to be kept manageable so that there is still profit. Return on investment is defined by profit and investment. Precisely, it is the ratio between the net profit and the cost of investment. Every investment in every aspect of businesses are held accountable for a high return on its own investment, and testing is no exception.

Testing is inevitably important in software development, but at what point should businesses be in favor of automated testing instead of manual testing? This question brings out the importance of the return on investment (ROI). The need to calculate the ROI for testing is essential, but how can it be done? Testing is a process that does not create products and with respect to that, no profit related to that; however, it has its costs. Since it does not directly create any profit whatsoever, how can it be measured something else other than 0? Testing does influence profit, but it is almost immeasurable. There must be certain parameters chosen to determine the profit provided by testing. But what can those parameters be?

A software product is directly affected by the size of the business, due to the development and testing needs. With a larger number of developers, a more advanced version of a product can be developed, seeing as two heads are better than one. With a larger number of testers comes much more developed and complex test cases both devised and run. So, the number of testers, the number of test cases are correlated to the product, and therefore, to the ROI.

Before going on to the other parameters that must be affecting the ROI of Test Automation, some definitions must be provided. A test case is an experiment of actions and events to check the behavior of a software. Actions are put in motion along with the expectation of specific events. Test cases are written every day, both in manual testing and in automated testing, to improve coverage. They can also be modified at certain times to cover new changes to the software.

ROI is calculated with the formula above. Monetary gains are harder to calculate since testing does not directly affect the product, but its quality. For this reason, the calculation is going to be done based on the time spent on the testing phases. Profit is the manual test execution and analysis time multiplied by the total number of test cases (automated and manual). Meanwhile, investment is the automated test script development time, automated test script execution time, automated test analysis time, automated test maintenance time and manual test case execution time.

Calculating the profit is much easier than calculating the investment since there are multiple parameters that contribute to it. But that does not mean that the investment becomes higher than the gain. Because automated testing makes more drastic changes. In every part of an investment, automated testing is considered and only one part manual testing is considered, and in the gains, not only automated tests are considered but also manual. Increasing the automated tests/testing is surely much more costly initially, but the way it works is that it replaces the norms that are manual testing, and with that, the investment on manual testing is reduced drastically. The gain does not change much initially, except with automated tests, you can continuously run them, even when people are not actually at work. Let’s consider that there are 10000 manual test cases and no automated cases, with an average of 8 minutes of manual test execution time. Gain = 8 mins * (10000 + 0) / 8 hours = 80000 minutes / 8 hours = 166.7. Now, instead, let’s replace 2000 of those manual cases with automated ones. New Gain = 8 mins * (8000 * 2000) / 8 hours = 166.7. There is no change (except the average time may change, so, minor changes are bound to happen). Before we calculate the investment, some parameters should be explained.

Automated test script development time is the time allocated for the test script development. Initially, this will be high, but when a high ratio of automated test cases is reached, it can be reduced. It can be calculated by multiplying the automation time per test and the number of automated test cases. Automated test script execution time can be calculated by automated test execution time per test multiplied with the number of automated test cases. Automated test analysis time is basically the same as the manual test analysis time, in that it is just the test analysis time. Automated test maintenance time is the usual maintenance time, and as expected, Manual Test Case Execution Time can be calculated by manual test execution time multiplied with the number of manual test cases.

Each parameter contributing to the investment is related to the automated tests except for the last one. Let’s use the same example, and let’s say the variables are as given below.

Automation time (per test) = 20 minutes

Number of automated test cases (in a month) = 250

Automated test execution time (per test) = 2 minutes

Test Analysis time = 60 minutes/day

Maintenance time = 90 minutes/day

Manual test execution time = 8 minutes

Manual test cases = 8000

And cases that require manual labor have to be done during the working hours, then let’s consider that to be 8 hours. Automated tests can be run 24 hours continuously, but malfunctions may occur, so we will say that they are run 20 hours a day.

Before the automation, the investment would just be equal to the manual test case execution time, and that would be investment = manual test case execution time = (8 mins * 10000)/8 hours = 80000 mins / 8 hours = 166.7

In the first month, the automation is being done, the investment would be:

This may seem like the investment is increasing, and it is, at least for some time. But let’s see what the investment is after the automation is done, up to where 2000 automated tests are reached:

At the point where a 20% automated test process is reached, the investment is reduced by 11.56% with respect to the initial investment. It is not too realistic to think of a 100% automated test process is possible, but considering an 80% automated test rate and having a number of automated test cases that is 100 per month (since at that rate, test script development would have decreased), the investment would then be:

And this result at an 80% automated test process is 30.69% of the initial investment. So, allocating a few months of development to automation does take away some investment, but in the end, the rewards are extraordinary. This is just the calculation of the investment for time, but the results would be similar in other ROI calculations as well. Getting back to the ROI calculation, the results of the example can be seen on the table below.

This kind of calculation of ROI according to time allocated does however make a few assumptions, one of them being that the automated test cases fully replace the manual test cases, because there are some tests that may be flaky and there are some test cases that require manual intervention; another is that the manual testing only counts one manual tester. Although, with automated testing, there can be multiple agents testing continuously and for a cheaper price than would be given to manual testers.

It is also important to address that with only manual testing, there may be cases which go undiscovered before the version release, and it may result in a post-release bug fixes that need to be released as well. But with automated testing, increased coverage can be reached. Automated tests can be run repeatedly to make sure there aren’t any mishaps occurring, meanwhile manual testers can do the manual tests over and over again, but the time it takes is a lot more than automated testing time. These advantages need to be considered as well for the calculation of ROI, but there is no unit of quantity with which these advantages can be measured.

Since there are multiple factors that alter the ROI calculation and not a lot of them are exactly measurable, it is quite difficult to calculate ROI for automated testing. There is no single correct equation to calculate ROI, and there should be room for error for the parameters that are not counted for. And to improve ROI, there should always be room for improvement, both for manual testing and for automated testing.

Start test automation today and save time and money on your investment. Do not forget that testing automation tools may be too complicated in the initial phases and that it may slow down the process, however, in the long term it can save you a lot of time and money only when executed properly. Therefore it is crucial to find the right tool to use that can be easily integrated into your system and used effectively. Have a look at what we can offer you with our products on testinium.com.