Test automation is the software process of automating repeatable tasks with a predictable outcome. The outcome shall meet predefined requirements, also called acceptance criteria. Automated software testing allows one to test more efficiently without the need to repeat manual steps to check that a piece of software behaves as expected. However, automating tests requires designing the test scenarios and the infrastructure needed.
While writing potential automated unit and integration tests is a task that developers and QAs typically perform, the test automation process needs collaboration between different team members.
For example, a software architect could define what a test framework should look like and its scope. An automation tester would be in charge of implementing tests executed without human interaction. Finally, a business analyst would be involved in the automation testing process to verify the system meets the business needs.
Agile practices have been adopted in much of the industry since the publication of the Agile manifesto in 2001. These practices include short feedback loops (one of test automation's most significant benefits) and validating software pieces fast.
Manual tests are prone to errors, which is another point to note. When automating a test, we must ensure that each execution will perform the same as the previous one.
We could wrongly think that automating a test is always a good choice, but that's only partially true.
Before explaining myself, I want to mention the test pyramid briefly (https://martinfowler.com/bliki/TestPyramid.html).
The test pyramid is a metaphor for grouping different types of tests. Unit tests that are developed and executed quickly will usually stay at the bottom of the pyramid. Whereas end-to-end test cases that are expensive to implement and deliver will sit at the top of the pyramid.
For this reason, unit tests are the first target when automating tests because they are cheap and save some time. Nonetheless, it could be completely different when we talk about end-to-end test cases.
First, we need to invest time to prepare the test environments and implement the test cases themselves. Later, executing these tests is time-consuming, so we would need to spend time on that. And finally, maintaining these takes a lot of work. And that is the reason why we consider keeping manual some of the most expensive test scenarios.
How, first of all, doesn't start with a W. However, it's an essential piece of the puzzle!
How is the most challenging question to answer because it depends on many factors. First, we should choose a testing framework based on the granularity of the test and the type of technology. Some examples of testing frameworks are Jest, GoogleTest, and Cypress.
If none of the existing market solutions fit your business needs, you could think about implementing a test framework, which adds more complexity. (We can explain more about this with our QA and Solutions team over a Discovery Workshop at Spark - book one here.)
Once the problem of the test framework choice is solved, we need to implement CI/CD strategy and again choose a tool, define standards, set restrictions, etc.
In the end, the possibilities are infinite. And it depends on the characteristics of the project, perhaps allocated resources or deadlines.
When it comes to actual projects, at Spark we automate our tests whenever possible. If we talk about unit testing, test automation is a must. But we must try to find a balance when we speak about integration or UI testing. It is not worth anyone's time to spend a week automating a test case that will be executed once per month. In any case, we reevaluate each approach periodically because the best solution in the past means something other than being the best solution in the present.
Regarding the automation tools we use at Spark, and quoting the words of our senior QA, Igor: "we are not slaves of a particular tool". Every QA engineer has preferences, but the final choice depends on the project.