We talked about what we perceive as the cost of unit tests: slowing us down.
That does not mean that unit tests are free. There are actual costs associated with unit tests, although they may not be what you think.
I want to start with the cost of writing unit tests. But not because writing tests (unit tests, integration tests, etc. ) slows you down – any line of code slows you down. All code, including unit tests is liability. As we’ll see, almost any cost that is associated with code has a unit test counterpart.
Refactoring
Like everything we write, including this blog post, the first version is not so good. The principle applies to code and unit tests. Because it’s not so good, you’ll pay later, and the cost goes up linearly, and sometimes exponentially, with how crappy it is. If you copied lots of setup code, when you’ll redo it, you’ll go through that setup code and try to understand what you needed from it. If you’ve picked lousy test names, you’ll need to fix them, after you found out what that unit test actually did. Refactoring is in an investment, so it has a cost part. The closest you do it to the coding, it will be cheaper.
Debugging
When code doesn’t work, we need to debug. Of course, that’s why we have unit tests for, and the amount of debugging would be less than debugging a full system. And yet, if our unit tests are obscure, rely on big setups, the code they test is complex, and has some evil singletons hiding in the bushes – we’re in for some fun debugging time.
Oh, and when they fail inconsistently? A full on debugging party.
Unit test bugs
Of course, we can’t really talk about debugging without mentioning bugs. It is possible, and surprisingly easy, to get bugs in our unit tests. It is also very easy to identify and solve these bugs with code reviews, but still. The cost of having bugs in our unit tests starts with debugging, but even more so, it introduces a giant fear factor into our life. We start asking: If there’s a bug in that unit test, what about the others? Can I trust these unit tests? And what about that moron team member of mine, whom I don’t trust at all? Should he even be allowed to write unit tests?
Not trusting our unit tests could be a tipping point on our quest for quality. It can topple the whole building.
Cross coverage
Some of the unit tests we write may be waste, because we have other tests covering that functionality already. That happens when we have bad factored code, or that our unit tests cover too much. Or you may just “want to be sure”, so you add a couple more unit tests. Any of these additional tests can have bugs, require debugging require refactoring. See how all of these accumulate?
This is just the opening show. There are more costs coming. Stay tuned.
0 Comments