So you’ve read part one. You see the point of building Extremely Fast Feedback Loops and you want to understand what it costs to have such a test suite. Let’s explore some problems…
The problem with testing in this way is that you must architect your system to support it.
While I do not have the figures to back this up, I have observed that the number of teams that practice disciplined TDD is a small figure. This is something that we at Made Tech are actively trying to change. On top of this, the number of teams practising Acceptance Testing with TDD (ATDD) similar to how it is described in the book “Growing Object-Oriented Software Guided by Tests” (an awesome book by the way), is also a smaller number still.
What this means is that systems built with ATDD, specifically one with Extremely Fast-Tests, will have an architecture that does not look like a “standard project”. This is the cost of Extremely Fast-Tests – the architecture will include aspects whose purpose will not be immediately clear to those unfamiliar with the challenges of building a system with Extremely Fast Acceptance Tests.
That is not to say that the architecture is not simple, but a different simple.
This is clearly an issue – it can cause confusion between groups of people with the same goals, but a different idea of what a good path to those goals looks like. It can cause tension in code reviews, it can cause distrust. I have heard comments such as “Why didn’t you just do it the way everyone else does it?”, “This could have been a few Rails controllers…”, “This seems way more complicated than it needs to be”. This kind of conflict is caused by only a slightly different perspective on how to approach testing software.
Another anecdotal observation is that projects built without practising ATDD tend toward the more complex. This is an observation of teams attempting to build systems that have a Hexagonal Architecture, without practising ATDD (or even TDD). What usually occurs is that programmers introduce more code (and complexity) to cope with the lack of high-level tests, or they find that their unit tests solidify an architecture and hold back evolution. With Extremely Fast Tests: teams have the ability to quickly refactor anything. This frees up teams to discover new architectural designs, and really evolve codebases, without worrying about breaking functionality.
If you want to picture a good architecture for this type of testing, imagine one where the Database Schema can be tackled last. Remember earlier that we mentioned testing below the Framework? Many programmers will be surprised by this, since the Framework, and also the Database, are both often seen as the core organising component. What ATDD does is make the Use Cases the core organising component, with the Framework, the Database and other external dependencies plugins to your Use Cases – the actual business logic. This type of architecture (out of the scope of this article) is known by many names including Boundary-Control-Entity by Ivar Jacobson, Hexagonal Architecture by Alistair Cockburn and Clean Architecture by Robert C. Martin.
We isolate the framework because it is a dependency that will only make writing your acceptance tests more difficult. Since most teams do not build software using disciplined ATDD, you will find that the many frameworks that language communities love (including C#, Java, Ruby), will provide minimal help, or require you to delve into arcane features to achieve this type of testing. This phenomenon is often described as “fighting the framework” – what we do to win the fight, is to not fight at all.
If you are using a compiled language – such as C# – your Acceptance Test assembly should not reference any MVC Framework. If you are using a dynamic language – such as Ruby, you should be able to execute all your Acceptance Tests without a single call to require ‘rails’ (or anything with similar effect) anywhere in your test suite.
Once you isolate the framework, and the associated code – your controllers and views, you can begin to test this aspect in isolation (without any business rules). We call this aspect the “Delivery Mechanism” because it is what delivers your features to your customer. By decoupling this aspect you can write extremely fast unit tests for every aspect of your Delivery Mechanism. This is because you can swap out the Use Cases with test doubles! This is the reason we do not write Acceptance Tests to test “through the Framework”. We want to enforce this isolation.
A neat trick is that now you can make your Acceptance Tests run at varying levels of abstraction e.g. you could have an Acceptance Test that is repeated twice: once “Subcutaneously” and once “through the UI”. What is happening are doing here is decoupling the steps of Arrange, Act and Assert from how they are executed within your system.
Whether you choose to build a system in a way to practice all the techniques described in this article not… If you want Extremely Fast Feedback-Loops, you will need an architecture to support it. Don’t be surprised if this architecture leads down a path of doing things that aren’t well supported by the wider programming community. Remember that your ATDD discipline puts you in the minority, and always consider the pros and cons of making potentially surprising choices. If in doubt, write an Architectural Decision Record for future maintainers. If you experience resistance from fellow programmers, perhaps consider offering to pair program with them.
We practice ATDD because we’re in it for the money – this is known as the “Money Premise” – we believe that ATDD enables us to deliver more value for less money than without. In reality, the Fast Feedback-Loops is what enables us to save money – anything that reduces the time it takes to go from an idea into being able to see the result of that idea (and crucially being able to validate it is a good idea) is usually worth trialling.