Unit testing Anti-pattern – Not Asserting
|This series goes through anti-patterns when writing tests. Yes, there are and will be many.|
|TDD without refactoring||Logic in tests||Misleading tests||Not asserting|
|Code matching||Data transformation||Asserting on not null||Prefixing test names with "test"|
You’d think that we don’t need this kind of post in our day and age, but I still see tests written without the assert part.
This pattern, as you’d expect, appears with beginners. However, when I ask, they usually know that they should assert.
So why do they still write tests like that?
A test without an assert (containing only an action with a possible setup) makes sure that the process works without an exception. In any other case, the test passes.
Crashing isn’t everything
I think that we can agree that knowing the code doesn’t crash is valuable.
In fact, if we expend the unit test into a wider scope (like integration, or an end-to-end test), it tells us that not only the functionality doesn’t blow up. The wiring, configuration and the whole path doesn’t blow up either. This is useful to know.
And usually this is where tests without asserts start.
We’re talking about beginners. They use their new found execution engine (read: unit test framework) for running a process, and see that it doesn’t break. The new tool helps with identifying that.
There is a deeper issue, though.
Beginners who start unit testing, usually don’t immediately rely on the tests as their only proof of working code. So the tests are written, after the code has proved to be working by running the application, or by other tool.
The process is: Write the code, check it works, and then write the test. When we write the test, we already know the code works. Now we have a test for it, one that doesen’t crash.
“Ok, boss, here’s the test you wanted me to write, and it proves that my code works, happy?”
Not only is the test writing extra work, now we need the assertion on top of it. Even more work.
You know what comes after the boss looks at the code, and mentions that the test doesn’t have an assert? Asserting on not-null. It’s the easiest to add.
And this is how we get working tests without assertions that are considered “ok”.
The right way
Much like assertion on a non-null return value, we can do better. While there is value in that kind of test, we value more having a specific assert that tells us where the problem is.
The value of the assertion, is that we specify the behavior we expect. Experienced developers start with how they will check the behavior, before setting up the test.
Even if it’s just one side effect of the operation, the assertion tells us what to look for, and what code caused that effect. Using triangulation we can pin point the problem and fix it quickly.
If we write the assertion we can learn that the code needs to expose information for the tests, and possible other clients. It can help with making code more usable.
If tests don’t have an assert, or if the assertion is on non-null, ask “how do you know it works?” in the context of the test. Once this question is answered, you can craft the assert accordingly.
Right, so you want to know a bit more about testing? Check out the “How do I test thee API?” and while you’re at it – check out the unit testing and TDD workshops.
David V. Corbin · March 1, 2017 at 5:55 pm
I completely agree with the root cause analysis… alas, breaking the pattern is much harder than identifying it…
Gil Zilberfeld · March 2, 2017 at 3:42 pm
It starts with people not getting inflicted with unit testing…
maciej wyrodek · March 2, 2017 at 11:51 am
Yeah assert not null is one of the worst. But my personal favorite is ‘Assert.IsTrue’ – maybe because I see it much often and actual value of most of this assertions is none.
Gil Zilberfeld · March 2, 2017 at 3:41 pm
Every one has his favorite!
Raja Nagendra Kumar · March 8, 2017 at 5:53 pm
Real issue is JUnit reports test methods and not asserts. I am sure, if JUnit and other test frameworks report number of asserts in each test method as way to report intial quality testcase, then this would be automatically fixed.
We (TejaSoft) has extended JUnit to report total asserts way back in 2002 🙂
Eido Askayo · August 26, 2018 at 4:53 pm
Thank you very much for this article.
Assume I have a method called “connect”.
This method returns a void, but throws exception.
Would you consider instead of return a void, to return a boolean, to avoid “no-assert” test? Is it answers “how do you know it works?” question you specified?
Gil Zilberfeld · August 26, 2018 at 6:28 pm
Thanks for the question. I discuss it in some depth in my Clean Code course. While it’s not part of the testing aspect, they seem to align
I consider the boolean (or any status return value) option less optimal. It requires the calling code to check for the result before continuing.In the exception case, the calling code needs a catch clause, it’s far below the usage method, so it doesn’t interrupt the reading of the code.
Now, if that’s the design choice, I would also test for throwing the exception, assuming that’s the behavior I expect. The test would read the way a calling code would use it. And it does use an assert (either in the form of @Expected/@ExpectedException or a try/catch clause in the test like in this post