This is the 2nd part of the series about analyzing what we’re coding, and therefore which unit tests and integration tests to write. Last time, we talked about the problem we want to test (either unit tests, integration tests or system tests) and a solution design. Once we have those, at least in some broad strokes, we can continue.
Step 3: Test categorization
At this point, after we’ve got a list of test cases and a design, we’re ready to think which cases we actually want to cover, and in what way. Buckets usually include manual system tests, automated system tests (from the UI), automated integration tests (from API and subsystems) and unit tests.
There’s no point in covering every possible case by every method, that would be wasteful. So we need to choose how we test the different cases.
While automation is more likely to be chosen over manual (for regression testing purposes), there’s a cost associated with automating everything. We usually automate the UI happy paths, automate unit tests around the logic, and integration tests where they provide value. We specify what’s left to be tested manually.
Like most of the steps in the process, it benefits from having a 2nd person to collaborate with. A tester, who has a wider understand of the system would make a great reviewer.
During this process, and after it, we’ll probably think about more cases, and decide some of the cases are very low risk, and therefore won’t be tested. Remember to collaborate and review your decisions with others, and not just decide alone.
Step 4: Testability design considerations
If we’re doing TDD. this step serves two goals. We want to confirm that our selection of test cases, including the integration tests and unit tests we’ll write are in line with the design we had in mind.
In addition, in order to be able to test, we want to make sure we expose the things we want to test. That means that our code needs to have seams we can mock, or insert probes so we can check our code works.
If we’re writing unit tests in a test-after manner, we want to achieve testability so we can write the unit tests correctly. If the design does not support it, we will need to do modifications for the design so it becomes testable. Check out the refactoring legacy code for testabilty series, but don’t jump the gun. We’re not touching the code yet, just analyzing.
Note, we now define “testable code” in correspondence to how we want to test it. Code can be monster-ugly, and still be ok for testing purposes.
Don’t miss a step with the next post, where we discuss design constraints..
Also, check out the video of a my talk “Unit Testing Strategy“.
0 Comments