Best Practices in Unit Testing

von am 18.11.2013
3
A metaphor for unit tests: silver cube out of which little blue cubes burst.

Unit-Testing Basics

Just like the term suggests, unit testing is about testing individual software units. Depending on the overall size of the software project and the programming language, the units in question are usually either single source files, single classes, or single components.

On a purely theoretical level, one could write unit tests without tool support: If you wanted to test a single C source file, for example, you could simply write a C program that includes the respective header file, defines a main function that calls a number of test functions, and implements these test functions in a straightforward fashion (by calling the functions under test). In real-life projects, however, unit testing is hardly ever done this way. Instead, we employ unit testing frameworks that provide some infrastructure and convenience functions.

Nearly every unit testing framework organizes tests in the following fashion: Tests are implemented by functions that are marked as test cases in some way (via C# attributes, C/C++ naming conventions, or the like). Tests are grouped in so-called test suites. Every test suite defines a setup function and a teardown function. These are invoked before and after every test function, respectively.

What to Expect

A proper test function states expectations. This is usually done via specific assertion functions or macros (depending on the unit testing framework). As soon as an assertion in a test case fails, the entire test fails. If all assertions are passed, on the other hand, the test succeeds.

Unit tests should always pose expectations derived from requirements or interface contracts — as opposed to expectations derived from the actual behavior of the code, from what the developer guessed should happen, or from other invalid sources. After all, the goal of tests is to check whether the implementation adheres to the specification.

Test Code is Code, Too

All the rules that apply for regular production code hold for unit test code as well. Example: Just like every function in your code should do exactly one particular thing, every test function should test one particular thing.

Always keep in mind that others might have to read and maintain your test code in the future.

Similarly, test code should be organized in a very clear fashion. For instance, a test can often be decomposed into several steps, and these steps should be indicated either by the way you write your test code or by appropriate inline comments. Typically, a test first needs to put the unit under test into a certain state, then perform a critical function call, and then verify that the outcome of this call meets the expectations derived from the specification. It is a good idea to let future readers of the test code know when each of these stages begins, and where the expectations come from (unless this is blatantly obvious).

As a rule of thumb, you should also separate tests for regular cases (where the unit is brought to a healthy state and its functions are called with valid arguments) from tests for error cases (the unit is in an error state and/or its functions are called with invalid arguments). Mixing the two leads to test cases that are hard to read or maintain.

Take the Cues (I)

Imagine you are writing unit tests for some software unit and it turns out that this task is cumbersome and unpleasant. Maybe it is hard to put the unit into a state in which you want to test its functionality, maybe it is hard to state meaningful sequences of function calls for testing. When this happens, do not blame the fact that you are asked to write tests, and do not blame the tools or frameworks in use.

Instead, take the cue that the interface of your software unit is hard to use.

This may be a very valuable hint — it allows you to redesign the interface during development and make it easier to work with for both you (as a unit tester) and about everybody who has to use your software unit in the future!

Dependencies

In many cases, it is not entirely simple to isolate some software unit A for unit testing — simply because A depends on other units, say B and C. There are two ways to get around this.

The first way is often seen in practice: One simply includes the units B and C in the unit test of A in order to satisfy the dependencies. Unfortunately, this is a clear violation of the definition of unit testing — i.e., this approach can be considered utterly evil in general. On the other hand, there may be justified exceptions (e.g., if B and C are fundamental infrastructural components — say, stdlibc replacements in a C/C++ project — without which unit A becomes entirely meaningless).

The second — and proper — way is to satisfy the dependencies by mocking: Instead of providing the real implementations of units B and C to the unit test of A, one replaces the required functions by mock implementations. The good news is that there is a plethora of mocking frameworks available, namely frameworks that allow you to replace the real functions with mock implementations in a convenient fashion.

Take the cues (II)

Imagine you are writing unit tests for software unit A, which depends on software units B and C, and it turns out that mocking the functions of B and C is cumbersome and unpleasant (“mocking hell”).

Again, this is a valuable hint that the interfaces of B and C are hard to work with for both you and everybody else. Consider a redesign of the interfaces of units B and C.

Code Coverage

The idea of code coverage is simple: You want to measure what portion of your production code is being “visited” by your unit tests. There are many different measures to choose from: function coverage (which portion of your functions are invoked by your test code), statement coverage, branch coverage, path coverage, condition/decision coverage, et cetera.

However, there is a problem with all these flavors of code coverage: They are metrics, and therefore, you will inherit all the anti-patterns that organizations tend to run into when they apply metrics. It really is as simple as this:

Coverage rewards people for hitting code in any way, but it does not particularly reward them for hitting the code in a meaningful way.

In the context of unit tests, this usually results in two consequences: Firstly, developers will have to switch from black-box testing (does my unit show the specified behavior on its surface?) to white-box testing (how on earth do I make the test code run into this nearly unreachable piece of code in line 155?). Secondly, developers are tempted to write test code that visits as many parts of their production code as possible — no matter whether this is bound to proper expectations or just done for its own sake.

In other words: Blindly following some metrics without any common sense is never constructive, and hence, the same holds true for code coverage. In fact, it is a well-known fact among software professionals that aiming at a goal of, say, 95% code coverage can be extremely helpful whereas hitting the very few last percent comes with large effort and questionable gain.

Unfortunately, a code coverage of 100% is required in certain domains (for instance, see the IEC 61508 standard). In order to suppress bad practices of reaching this goal as outlined above, it is imperative for a software team to put the unit test code under review.

Conclusion

Unit testing is an endeavor that is often underestimated; it definitely takes a senior software engineer to write senior-level unit test code. Therefore, do not treat unit tests as a mere by-product. Provide your team with proper guidelines and help, make the management understand that you will have to assist junior developers with their test code, and help everyone avoid the pitfalls mentioned above.

P.S. QUDZWU9GYASU 

Einen Kommentar hinterlassen

Ihre E-Mail-Adresse wird weder veröffentlicht noch weitergegeben. Pflichtfelder sind mit einem * markiert.

3 Kommentare für “Best Practices in Unit Testing

  1. Justin Kendall

    I am an aspiring software engineer and game programming hobbyist, and I've found several of your articles informative and helpful in my understanding of many of the industry's core principles. Thanks.

  2. Pingback: Duck and Cover: Why to be Careful with Claims about Code Coverage