This series goes through anti-patterns when writing tests. Yes, there are and will be many.
TDD without refactoringLogic in testsMisleading testsNot asserting
Code matchingData transformationAsserting on not nullPrefixing test names with "test"

Catchy little title, right? Turns out it’s important for unit tests.

Once people understand they actually need to write an assert, this pattern appears almost immediately. (And for those who think: Who doesn’t write an assert, wait for the next post.)

Consider this test:

@Test
public void convertXtoY_OnSuccess_NotNull() {
	X x = new X();
	Y result = x.convertXTo(Y.class);
	assertNotNull(result);
}

The conversion works, obviosuly. The test proves it.

First let’s discuss the acceptance criteria. Is a null result indeed a symbol of failure?

Because, if it’s not, and the code always returns a non-null value (for example, if using the Null Object Pattern), then we have ourselves a test that always passes, and that’s not much of value.

But for the sake of argument, let’s assume that null means failure.

So it’s ok to check for it’s non-null-ness, right?

Yes.

And, also not really interesting.

Because the client code probably is more interested with the non-null returned value. It will probably do something with it, like:

void DoSomethingWithY(X x) {
	Y y = x.convertTo(Y.class);
	doSomethingWith(y.getValue());
}

Because that’s what we do with results of functions, we use them for something.
Now, if the client code is going to use that, why not the test? The test is a primary user of the code, just like production code. Then why not write the test like:

@Test
public void convertXtoY_OnSuccess_ReturnsTheValue() {
	X x = new X();
	Y result = x.convertXTo(Y.class);
	assertEquals(theValue, result.getValue());
}

Well, sure, that’s better.

But what if it actually returns null?

If the result is null, we’ll get a nice NullPointerException (java here, but depends on your language of choice), and that exception would point to the exact line where the assertNotNull would point. It’s like magic.

So we didn’t lose information, and instead, gained the confidence on how the production will acutally going to be using our object works.

My guess – we’re better off.

Check out the following workshops where I talk about errors in unit tests:

Categories: anti-patterns

2 Comments

David V. Corbin · February 22, 2017 at 5:11 pm

There is a flip side. Having (only) one test that captures multiple types of failures has a number of downsides. Using your example, it is likely that the code path that returned a null is different than the code path that would return a “wrong value”.

I often play a game with development teams I am mentoring. They write their tests (or have written them previously). I make a breaking/bad change to the source code. They are given a list of the tests that fail and are then challenged to identify what I did to the code.

As systems become more complex (i.e. testing progresses through a hierarchy from unit to system tests) this technique becomes ever more valuable.

    Gil Zilberfeld · March 2, 2017 at 3:47 pm

    Interesting. I might take up that method. Thanks David!

Leave a Reply

Avatar placeholder

Your email address will not be published. Required fields are marked *