As far as I can tell, the test suites in most if not at all software projects with a team working on them gradually descend into “fixture hell”. Most software engineers probably recognise that situation. It’s when the general set up and fixture management for the test suite has got so tangled and out of control that no-one dares to try and sort it out. Instead it turns into a vicious cycle where for each new piece of work there’s little choice but to make the fixture hell even worse.
The situation is a bit like an abstraction castle but in the test suite instead of the implementation. The main difference with an abstraction castle is that fixture hell tends to develop organically over time and with the contribution of many different people.
I don’t think this is really a discipline problem, though, and I doubt a convention could fix it in most cases. Setup gets shared because writing it out again is time-consuming. Teams tend to try and share test setup roughly in proportion to how time-consuming it is.
If we can make it cheap and easy to set up test state, it can mitigate that slide into ever more complex and costly test setup.
So the thing worth working on is the cost and complexity, and that cost is mostly a property of the tools the test has to use. It often sits outside the immediate control of the person writing the test. For example, a developer might need a customer entity with an order attached, and find that the only route to that runs through a complex web of test fixtures.
One way to reduce that complexity and make test setup cheaper is to ensure that the pieces involved are independent. Each piece of test setup should stand on its own, inside the scope where it’s used. Ideally all you need to do is instantiate a class or call a function, and there are few if any side effects.
The @kensio/yulin
and
@kensio/part-factory
open-source packages both address this problem. In Yulin, new SimAws() instantiates a fresh
simulated AWS with isolated state of its own. In Part Factory, a factory is a module-level constant
whose make() method returns a new object. Both are cheap to call and can work inside a single test
case, independently of the rest of the suite.
That approach leaves a test free to build up what it needs in place, in the order it needs it:
it("archives orders above the reporting threshold", async () => {
// Given a simulated S3 bucket to archive into.
const simAws = new SimAws();
const simS3 = simAws.s3();
await simS3.createBucket(new CreateBucketCommand({ Bucket: "foo-archive" }));
// And given an order above the threshold.
const order = orderFactory.make({ total: 5000 });
// ...
});
Every part of that state belongs to that one test and goes out of scope with it, so reading from the top tells you the whole situation the assertion is eventually made in.
There’s still plenty of room for extracting steps. I’ve written before about naming each step as a function beside the test that uses it, which is often the best way to keep a longer setup readable. The distinction is whether the step belongs to the test or the test belongs to the step.
An unexported helper sitting in one file can be pulled back inline if it hurts readability. Conversely, a fixture that a large part of the test suite is built on is a much harder thing to untangle. The tangling of fixtures and test setup often becomes a vicious cycle.
I’ve also argued before that it’s usually worth erring towards more tests rather than fewer, which is the other side of this. Writing one more test is worth it when setting up its state is easy. Where the state is expensive or complex, the test suite often becomes less and less effective at its original purpose of raising quality and maintaining velocity.
