When developing rich software applications, a crucial aspect of your work is testing – and some argue that testing should not even be regarded as a separate stage in the process, as it’s intimately related to (and chronologically merged with) the development itself. There are many testing levels (such as unit testing, system testing etc.), and also there are more testing methods available (for example: white box testing vs. black box testing vs. grey box testing), making it hard to decide over the testing strategy to be used in one situation or another.
A popular testing methodology is manual testing. This simply means manually testing the software, on various functional paths / scenarios. Contrary to manual testing is test automation, which requires the use of a piece of software to run the tests and validate the results. Practically, the software (and not the tester) compares the resulted outcomes with the predicted ones.
Each approach has its own advantages and disadvantages, but in the end the choice of using one of the methods or the other depends entirely on the type of software you are developing. But let’s see below the main differences between manual and automated testing:
Who does the testing
Manual testing requires a tester to play the role of a typical user and to employ all the software functionalities in order to ensure a correct behavior.
In the case of test automation, a tester / developer has to define / write a series of tests, and then have the test automation software running those tests, and comparing the obtained results with the expected ones.
How does the process work
When testing manually (according to a “scripted testing” procedure – meaning that the tester follows a certain test suite), the tester will need to write a series of test cases (which will comprise a test suite), and then run those test cases, one after another. In some cases, the test suites are not written by the tester, but rather by a testing supervisor or quality assurance engineer – or even by a developer.
And besides the scripted testing, there’s also the so called “free testing” (formally defined as exploratory testing), which simply means that the tester tries to use the software in any testing scenarios that he can come up with, on the spot, and write down any bugs that might show up on the way.
Automated testing implies almost the same process as the scripted testing, just that the test cases are automated (written for and ran by a machine). Usually the test cases are written by a developer, rather than a tester.
How much time does it take
In the case of automated testing, as the actual tests are ran by a computer, rather than an individual, the time invested in the overal testing process will be reduced (as compared to manual testing). On the other hand, authoring the test suite can be quite time consuming, and the same goes for the time spent interpreting the results.
One caveat here: unless you can reuse the automated tests (for future releases that contain the same features, or for future regression testing), test automation can be just as time consuming (as a percentage from the overall effort) as manual testing.
How much does it cost (apart from the time invested)
Test automation involves higher initial costs than manual testing, as it requires tools, environments, test development etc.. If the automated tests can be reused, then the costs are lowered.
When can it be used
A product needs to be stable 50% – 60% in order to use test automation. On the other hand, manual testing is always there, and can be applied at any stage of the development cycle.
Automated tests are suitable for: regression and confirmation testing, random testing, performance and reliability testing, and when the results can be interpreted quickly. Manual testing, on the other hand, works best for: installation, setup, and maintenance testing, configuration and compatibility testing, testing for errors, localization testing (only a specialized tester can decide if a translation doesn’t has sense, or if it’s offensive from a cultural point of view, or inappropriate), usability testing, documentation.
Of course, there are situations when tests can be done either manually or automatically, with certain benefits for each approach. Such situations are: functionality testing, user scenarios, user interface testing, date and time testing etc..
So there is no universal answer, whether to use manual vs. automated testing: you can choose to use one or the other, depending on the software that you are building, also depending on the situation and the stage of the development cycle. When the complexity of the software requires human skills and judgement, then you have no choice but using manual testing – however if you need to repeat a simple testing cycle many times, then automated testing will save you time.
Because software products are dynamic and they tend to change constantly, your testing strategy and the test suites will need to be updated as well, to keep up with the updated functionality – therefore you may even end up going back and forth between manual and automated testing, for the same product, as time goes by.
