The Business of Test Automation

We take pride in our extensive adoption of test automation. We see significant business value and commercial advantage both for us and our clients in providing automated test suites across our deliveries.

We understand that computers are more efficient than people at running repetitive, and usually monotonous, tasks and look to harness this efficiency to reduce the amount of time it takes to make a change and be confident enough in this change to deploy it to a production environment where it can start delivering value (‘cycle time’).

What do we mean by Test Automation?

When we talk about Test Automation, we mean that alongside programming the functionality behind a project, we also program tests that check that this functionality is working. Think of it as double-entry bookkeeping for programming.

These tests that grow in number as the project is built out can then be run at the click of a button to assert that any new changes haven’t adversely affected previously working functionality.

Automated and manual testing

In a more typical manual testing environment, time is often spent having testers (or on larger applications; a small army of testers) manually working through key user journeys ensuring they function as expected. Every time a change is made to the project, this functionality needs to be tested again, requiring a tester to repeat a test they’ve already run – perhaps tens of times previously.

We still see and derive value from performing an element of manual testing on projects. However, we believe the human investment in testing is better used in focusing on testing that interaction works well and that the application visually appears and works as expected – things that computers struggle to do as well as humans.

The cost of test automation

Incorporating test automation increases the time spent on a project during the early phases when a test suite is being built up. However, on a project of any longevity, we believe dividends are quickly paid. The ability to make changes with confidence that you haven’t affected part of the project that you may know nothing about is a significant advantage.

Software applications (and certainly the sort of applications we typically produce) are rarely treated as immutable. Changing software to meet new business needs and challenges is a constant; and we believe it should be encouraged and embraced. Test automation is one of the primary enablers in allowing confident ongoing modification of software to meet changing commercial requirements.

The cost of fixing software defects increase exponentially the later in the process they’re caught. Research suggests that a defect identified by a secondary tester costs 10 times as much as a defect that is identified by the originating developer and that identifying a defect in production costs between 10 and 25 times as much when compared to a defect identified by the originating developer. Empowering everybody on the team who is making changes with the tools to get early-and-often feedback should significantly reduce the ongoing cost of change.

Fast feedback

Fast feedback (we’d typically be aiming for a test suite to take less than 10 minutes to complete) allows developers to make large changes to applications, which we often see as necessary to deal with changing requirements, without fear that they’ve introduced adverse side effects elsewhere in the application. We see that automated testing also increases the general level of quality and maintainability of underlying application code.
Types of testingIn addition to automated acceptance testing, which we use to assert that functionality works as expected, we rely on automated tests to check for common security defects and to assert that we’re producing code that adheres to accepted style and complexity (or rather, simplicity) standards.

Test automation has been around in some form since the 60’s. As web applications become more business critical and become viewed more widely as true software products, an extensive range of tools have emerged to provide many different flavours of automated testing against the many different aspects of web application development.

We still see the adoption of test automation as surprisingly niche across the range of organisations that can be involved in the delivery of web applications. We’d anticipate that the significant commercial advantages afforded by test automation will show shifts over the coming years as the costs, both financial and in lost opportunity, of hard-to-maintain software cause challenges for more organisations.

About the Author