Over Exposure and Privacy Issues

Standard

This conversation comes up in every training I do about testing. I know exactly when it will happen. I know what I’m going to say, what they will say, and I know no one comes out completely convinced.

Then violence ensues.

We want to test some code. However, this code had started being written in the age of the dinosaurs, its size resembles some of the big ones, and the risk of changing the code is the risk of feeding a Jurassic beast.

There’s no point ignoring it, we’ll need to modify it in order to be able to test it. In fact what’s really bothering us is that some of the information in there needs to be accessed. I suggest changing the visibility of some members.

This escales quickly

Curious person: “What do you mean make it public”?
Me: You know, make it public. Or add a method that will enable us to set up the class easily, so we can test it.
Curious person: “But that means breaking encapsulation”.
Me: Might be. Is that wrong?
Mildly suspicious person: “Well, the design is there for a reason. The member is private because it needs to be private, we can’t expose it”.
Me: Remember that tests are first-class users of the code? That means that the design can change to support them too.
Mildly angry person: “But you can’t expose everything, what if somebody calls it”?
Me: That’s the point, that the test can call it.
Angry person: “But you can’t ensure that only tests will call it. At some point, someone will call it and create hell on earth. We’ve all been there”.
Mildly angry Me: Yes, it might happen, and we can’t predict everything and plan for everything, software is a risky business.
Very angry person: “THESE RULES WERE WRITTEN IN BLOOD!”

The design perspective

Let’s ignore the fact that you’re defending a horrible untested design, with the argument that changing the accessibility of one member will make it so much worse.

Encapsulation is a great concept. Having public and private members makes perfect sense. Some things we want to expose, others we want to hide.

But the water becomes murky very quickly. If we were talking black and white, only private and public, that would be ok. But for those special times, when private is just not enough…

There’s “friend” (C++).
Or “internal” (.net).
Or implicit package visibiliy (Java).
Not to mention “protected”.
Or reflection that bypasses everything.

It’s like we’re breaking encapsulation, but not really breaking it, so we feel better.

Let’s face it, design can serve more than one purpose, and more than one client in different ways. The language giving us tools doesn’t solve all the problems. That’s the cool thing about software, it can be molded to fit what we need it to do.

That involves living with not-so-perfect designs. It also means “changing the code just for the tests” is ok, because that’s another good purpose.

BUT SOMEBODY WILL CALL IT AND PEOPLE WILL DIE

Yes, somebody might call it.

That somebody might also look at the code and decide to change it, or add an accessor themselves.
It’s silly, not to mention risky, to think that a language keyword is the only thing standing betweend us and a disaster.

In C everything is public, how do you prevent the “someone might call it” disaster waiting to happen there? Playing with .h files and custom linking can be a lot more risky.

We do that the way that’s always more effective than the code itself: Processes that involve humans, not relying on tools.

We need to understand the purpose of code – create value for someone. If the code is beautiful but cannot be tested,  you don’t get points for using the “right” encapsulation.

The value comes when functionality works. In order for it to work, we need to check it. If we discussed the risks involved, and decided value comes from testing, it usually outweighs the risk of “someone might do something bad with the code” (after many have abused it already).

And if you feel this risk is not negligable – do something about it. Do code reviews, document and share the knowledge, create architecture guidelines.

But don’t rely on a language feature as the first resort.

Unit Testing Anti-Pattern: Prefixing Test names With “test”

Standard
This series goes through anti-patterns when writing tests. Yes, there are and will be many. 
TDD without refactoringLogic in testsMisleading testsNot asserting
Code matchingData transformationAsserting on not nullPrefixing test names with "test"

History teaches us that old habits are hard to break. Much like the proverbial monkeys who haven’t been there for the first electric shock, but still wouldn’t get near the banana, the habits are even harder to break because “that’s what we do here, we can’t explain why”.

People are still writing tests that contain “test” in the name, and you know why? Because that’s how they do it here. And it makes sense, right? It’s a test!

Which begs the question: we know it’s a test, but how does a test framework know it’s a test? What makes a test different than any other method?

The olden days

In Java, when JUnit started to take shape, and later in .net and other reflection supporting languages, before annotations were supported yet, the test framework needed to identify a test in some way, and until JUnit 3, that way was to start the test method with “test”. Convention over configuration. Beautiful.

Since JUnit 4 that took advantage of Java annotations (@Test), the requirement for a test to start with “test” was no longer needed. In NUnit, and other .net framework, attributes played that part.

In other languages that didn’t identify tests using reflection, things got more interesting.

Identifying the tests using reflection, saved the need for test registration. If you look at many C/C++ older frameworks, you’ll see the registration part, for example in CppUnit:

We manually need to register each test in the test suite.

There was no way the framework could diffrentiate between a test and another function automatically. In more modern frameworks, the registration takes place by macro implementation for you, so there’s no need to go find tests, and tag them properly.

By the way, another advantage of prefixing the test with “test”, helped with collecting and identifying the tests for registration. That’s not needed anymore either, due to the automatic registration done by macros.

To summarize, with modern frameworks there is no need for the identification of tests by prefixing them.

Furthermore, if locating tests is your issue, modern project conventions help with that more than a name can. Maven projects, for exmple, tell you where to place and find the tests. The project file architecture has much more to do with identifying and running test than anything else.

And so ends our history lesson, and you probably want to ask, again: Then why don’t people stop doing that?

Apart from the “that’s what we do here” part, I mean. They don’t have a good explanation.

Still there’s a good explanation to why you should drop the “test” prefix.

Visual noise

Code is complex enough already. Adding more code simply distracts us from understanding it. It’s true with big classes and methods, and it is also true with tests.

I always encourage to write descriptive test names (and with regular methods too). That means they get long names. Which is ok if they contain the information for better description of what the test checks, and at what context.

Obviously, having the word “test” doesn’t help me understand it better. It’s just noise, a distraction from the important bits.

This is just low noise. Annoying, but bearable.

But when you look at a CI report, and everything starts with “test”, it’s no longer low volume. It takes up a very big portion of the screen and information. Again, Everything on the test part of the CI report is obviously tests. The noise was amplified and distracts us further from identifying sibling tests and their results, looking for specific tests, etc.

In a nutshell: Prefixing tests with “test” doesn’t help anyone (unless maybe dealing with legacy test frameworks), and can even cause delays when looking to solve problems with failed tests.

Just don’t do it.

Unit testing Anti-pattern – Not Asserting

Standard
This series goes through anti-patterns when writing tests. Yes, there are and will be many. 
TDD without refactoringLogic in testsMisleading testsNot asserting
Code matchingData transformationAsserting on not nullPrefixing test names with "test"

You’d think that we don’t need this kind of post in our day and age, but I still see tests written without the assert part.

This pattern, as you’d expect, appears with beginners. However, when I ask, they usually know that they should assert.

So why do they still write tests like that?

A test without an assert (containing only an action with a possible setup) makes sure that the process works without an exception. In any other case, the test passes.

Crashing isn’t everything

I think that we can agree that knowing the code doesn’t crash is valuable.

In fact, if we expend the unit test into a wider scope (like integration, or an end-to-end test), it tells us that not only the functionality doesn’t blow up. The wiring, configuration and the whole path doesn’t blow up either. This is useful to know.

And usually this is where tests without asserts start.

We’re talking about beginners. They use their new found execution engine (read: unit test framework) for running a process, and see that it doesn’t break. The new tool helps with identifying that.

There is a deeper issue, though.

Beginners who start unit testing, usually don’t immediately rely on the tests as their only proof of working code. So the tests are written, after the code has proved to be working by running the application, or by other tool.

The process is: Write the code, check it works, and then write the test. When we write the test, we already know the code works. Now we have a test for it, one that doesen’t crash.

“Ok, boss, here’s the test you wanted me to write, and it proves that my code works, happy?”

Not only is the test writing extra work, now we need the assertion on top of it. Even more work.

You know what comes after the boss looks at the code, and mentions that the test doesn’t have an assert? Asserting on not-null. It’s the easiest to add.

And this is how we get working tests without assertions that are considered “ok”.

The right way

Much like assertion on a non-null return value, we can do better. While there is value in that kind of test, we value more having a specific assert that tells us where the problem is.

The value of the assertion, is that we specify the behavior we expect. Experienced developers start with how they will check the behavior, before setting up the test.

Even if it’s just one side effect of the operation, the assertion tells us what to look for, and what code caused that effect. Using triangulation we can pin point the problem and fix it quickly.

If we write the assertion w.e can learn that the code needs to expose information for the tests, and possible other clients. It can help with making code more usable.

If tests don’t have an assert, or if the assertion is on non-null, ask “how do you know it works?” in the context of the test. Once this question is answered, you can craft the assert accordingly.

 

 

 

Unit Testing Anti-pattern – Asserting on Not Null

Standard
This series goes through anti-patterns when writing tests. Yes, there are and will be many. 
TDD without refactoringLogic in testsMisleading testsNot asserting
Code matchingData transformationAsserting on not nullPrefixing test names with "test"

Catchy little title, right?

Once people understand they actually need to write an assert, this pattern appears almost immediately. (And for those who think: Who doesn’t write an assert, wait for the next post.)

Consider this test:

The conversion works, obviosuly. The test proves it.

First let’s discuss the acceptance criteria. Is a null result indeed a symbol of failure?

Because, if it’s not, and the code always returns a non-null value (for example, if using the Null Object Pattern), then we have ourselves a test that always passes, and that’s not much of value.

But for the sake of argument, let’s assume that null means failure.

So it’s ok to check for it’s non-null-ness, right?

Yes.

And, also not really interesting.

Because the client code probably is more interested with the non-null returned value. It will probably do something with it, like:

Because that’s what we do with results of functions, we use them for something.
Now, if the client code is going to use that, why not the test? The test is a primary user of the code, just like production code. Then why not write the test like:

Well, sure, that’s better.

But what if it actually returns null?

If the result is null, we’ll get a nice NullPointerException (java here, but depends on your language of choice), and that exception would point to the exact line where the assertNotNull would point. It’s like magic.

So we didn’t lose information, and instead, gained the confidence on how the production will acutally going to be using our object works.

My guess – we’re better off.

Unit testing is part of the job

Standard

In one of my unit testing courses we had a disucssion I’ve been part of what feels like a million times.

The question the team asked was: “How do you convince the client to pay for unit testing?”

Because it adds like 20% more to the project’s budget, right?

While we can discuss the validity of the number, the interesting thing is what lies behind the question about “the added cost of unit testing”.

It’s an add-on!

Imagine going to a client, and saying “we can do the project cheaper, we’ll just skip the design part”. Does this sound professional? Would any client accept a system that didn’t go through some kind of design?

Even when a client wants a POC or willing to accept a lower quality system, they expect that we’d design it. They will not consider us professional if we’d provide them something that wasn’t properly thought through.

We don’t talk about design as a separate, possibly patched-on section in the project, because we consider it a part of the job.

But, unit testing? Sure, we can add or drop it for the right price.

It’s not “production code”. It doesn’t even get delivered. It’s a novelty that people hear about in conferences but makes developer lives in the real world harder.

Whatever the excuse is, it becomes a separate line-item in the project plan.

If we continue talking about unit testing like an add-on to coding, it’s easy to see how we get to these conversations regularly.

Enough is enough

Unit testing is an integral part of development process. It is not something you drop to make the deadline. It’s part of the project, a part of regular development work. It’s like writing code, designing, testing, documentation – any other part of the project.

Writing code (sometimes horrible, complex code) is part of the job. It has its cost (writing, maintaining, reviewing, fixing). It has its value (working functionality).

Well, guess what. Unit testing is the same. It is one of the things we do to produce working software.

Unit testing is simply a part of the job.

 

Unit Testing Anti-Pattern: Data Transformation Tests

Standard
This series goes through anti-patterns when writing tests. Yes, there are and will be many. 
TDD without refactoringLogic in testsMisleading testsNot asserting
Code matchingData transformationAsserting on not nullPrefixing test names with "test"

Should we write tests for everything?

Absolutely not. Tests that do not help us, cause us more pain while writing and maintaining them are waste. Here’s a common example I see all the time. We have code that translates data from one object or structure to another. Like this code that takes a Person and transforms it into a Student object:

Very simple code. Also usually happens to be a patch-up code. It happens a lot when you have a working version of the code, for years. It’s working faboulesly without tests.

But now in v3, we need to add some functionality, and as part of the new design we need a subset of the information or some kind of translation of it. It sometimes includes logic, but a lot of the information just moves around fields.

Now, we can definitely write tests that check that the transformation was correct. Fill out a Person with our on data, then assert it got to the right place in the Student object. In fact, data transformation tests are the easiest kind to write, since the data doesn’t have any dependencies.

Easy test to write? Increase my coverage for almost free? WIN!!!

Hold your horses, cowboy.

You’ve written a test, and it passes. What will cause it to fail in the future?

What kind of person are you?

Are you a person that compiles everything every a few minutes, just to make sure everything is still in order? Sure you are.

You’ve learned in the past that red squiggly lines, or compiler errors are an early warning that will save you time later.

That’s true for tests as well, they are another layer of early warning. Only they don’t work at the compiler level. Instead, they check the logic in the code. That’s why we tend to write test for code with conditionals, forks in the code, etc.

Now, going back to our data transformation: If there’s no logic in there, when will the test tell us something is wrong? If the answer is never, that test is pretty useless.

There are a couple of example where we can mess up, that tests can catch and I’ll use the code example above to categorize them:

  • Mix up the age and street fields.

If you messed up different types of data, the compiler will tell you, or other tests start failing. Switching completely differnt types of fields is going to raise a red flag somewhere else, so tests for these fields are probably wastefull – the risk of the problem getting through and not identified is low. We’re left withthe maintenance costs of the tests. The tests will probably pass for the rest of their lives, so they just incur time instead of helping.

  • Mix up country and street fields.

This is a bit more tricky, as the chance of catching the error without tests becomes smaller. Compilers won’t catch it, since the fields are the same type. The semantics are important, and without unit tests, we can catch those with code reviews. However, higher level tests (integration, end-to-end) probably will catch them. If the impact of the problem is low (for example, we keep the address details, but communicate through email) we can skip the unit test in that case too for the same reasons as before. (And ponder the question – why do we need the address for? But that’s for another post).

  • Calculated the isAccepted incorrectly

That’s the riskeist one, since we got business rules riding on it, that may not be caught by other tests. We are transforming data that is important business information, andwe want to make sure that is transfered correctly. I might write a unit test for that, but then ask the question again: If this is so important, why don’t we have high level tests for it? Aren’t there cheaper ways to make sure we transfer the data correctly? (Like, you know, code review?)

So you probably don’t need unit tests for data transformation code. Unless there’s important logic in there.

What do we do then?

We have a couple of options (that you probably have thought about already):

  • Write a unit test the logic transformation (and leave out all the direct data passing uncovered)
  • Seperate the logic into another class or module, and test it there. Then the data transformation remains pure and clean, and doesn’t need any tests.

We are sometimes incentivized to get more coverage and we look at code that transforms data as quick wins. In fact these tests won’t help us in the future, and will just cause code lock-in, prolong the test run time and cause us maintenance repairs. As long as you have other ways of dealing with it, data transfomration tests you can probably skip.

 

Implementing Unit Testing – Outcomes

Standard
This series deals with the implementation of a unit testing in a team or multiple teams in an organization. Posts in the series include:
GoalsOutcomes

We’ve discussed what the goals of the implementation are, and now it’s time to talk details.

What do we expect to happen? When would we know we have achieved our goals? We’re looking for evidence that will tell us if the implementation has reached its goals.

However, the goal of the implementation is not the goal of the team. When the implementation reaches its goals, it has an impact on the team.. We’re are not doing it just for the sake of writing tests.

The first outcome we expect to see is quality improvement in the product. It’s true that better quality may not be impacted solely by unit tests (that holds true for the other outcomes too), but nonetheless it is an outcome we’re looking for.

So how does the implementation achieve the outcome?

If there are tests around buggy code, and the team continues to run them, and address test failure as an important issue, the bugs get fixed early. That leaves more time for wider and exploratory testing to be done, rather than spending time on the simple bugs that could have been caught earlier. We are better informed on what we release, and confident that the quality is good enough to release. As a side effect, there are also less fire fights and emergencies that keep us focused on quality.

Something we can actually measure as a result of this rise in quality are better “ready to test” builds. If we have an automated sanity suite (integration and/or system), we’ll see an increase in the number of builds that pass the sanity suite. That is simply because the unit tests find those early stupid bugs before they enter the testing cycle.

Product quality is one outcome; we can see also improvement in code quality.

While writing unit tests alone may not result in better code, the tests are just part of the program. Training and coaching also focuses on better code and design, not just as a focus for better and easy to write tests. If the material catches on, people will eventually write better code, and sustain it as a team convention. Code is going to be more readable, and with tests – easy to refactor, so people will improve their code.

Finally, we go back to the goal of the implementation itself. We do want to see the testing effort becomes a regular part of development. While it’s not the main outcome we’re going for, it is definitely a proxy of whether our implementation is successful.

Outcomes come out at the end. We don’t want to wait that long to see what happens. Next time we’re going to discuss what metrics we can track as leading indicators, to see if we’re on the right track.

 

Implementing Unit Testing – Goals

Standard
This series deals with the implementation of a unit testing in a team or multiple teams in an organization. Posts in the series include:
GoalsOutcomes

In this series, I’m going to discuss the strategy of how to roll out a unit testing implementation in an organization. As we all know, most unit testing initiatives start by developers, and if they are lucky, it gets picked up by their team, and maybe, in a viral fashion, go on to other teams.

This is all very dependent on how well the process goes, how people respond, and more importantly, how much management gets behind it.

But when it does happen, it is still fallible. So how can we support the process, in such a way that teams adopt it, and continue doing it?

That’s where this series starts. We’ll go through the different aspects of implementing the process. And we start with the end in mind.

GOOOOOOOAAAAAALLLLLLL!

Ok, maybe not that kind.

We want to know what to aim for, so we can adjust accordingly. As with any new process, the chances that everything goes according to plan is quite low, so keeping in mind what we want to achieve  is important.

  • Sustainable, continuous effort of unit testing.

Obviously, that’s the major one. We want writing tests become an integral part of development. And it’s not just about writing – we want to include the unit testing strategy as part of development. The sustainable part means that it’s not just part of writing software, it also doesn’t require extra nagging.

It’s just part of the job.

  • Tests are effective.

Yeah, we kind of miss that sometimes. Good, effective tests don’t just write themselves. Unfortunately, the tests we write first are pretty crappy, and those live the longest. We want that after testing becomes a way of life, we write effective tests, and get rid of the bad ones.

  • Quick on-boarding of new people on the team

Remember, the goal is after the implementation plan, so we want this goal to be attainable after the initial roll-out. When new people join the team, without knowledge of basic testing skills, and particular team knowledge, we want them to get on-board with less hassle as possible. This doesn’t just happen by itself, and we’ll need to invest time and resources to make that happen.

  • Easy on-boarding for a new team or group

If there are only a handful of developers, that’s not a priority. But if we want our 500 team organization moving towards developing tests, that’s an issue we should address. Note, that here the focus is on “easy”, not “quick”. We cannot expedite a new process implementation, it just takes time. We do want to be prepared, with examples, coaches, etc., to support a new team starting out the unit testing way.

That’s a nice set of worthy goals. Of course, you can add yours if they are as worthy.

Next time we’ll discuss how we know we’re there and what leading indicators we need to track.