Repository navigation
TDD Anti Patterns
This list was originally posted to http://blog.james-carr.org/?p=44, but is no longer available. I resurrected using the wayback machine
Posted on November 3, 2006 Recently I began to write a paper on TDD Anti-Patterns, and decided to first quickly jot down some of the most common ones that others or myself have encountered “in the wild.” I then posted what I had on the testdrivendevelopment on yahoogroups mailing list, and got an excellent bit of feedback!
As a side note, keep in mind this is merely a catalog at the moment, hopefully to be expanded in full down the road.
An entire unit test that passes all of the test cases it has and appears valid, but upon closer inspection it is discovered that it doesn’t really test the intended target at all.
A test that requires a lot of work setting up in order to even begin testing. Sometimes several hundred lines of code is used to setup the environment for one test, with several objects involved, which can make it difficult to really ascertain what is tested due to the “noise” of all of the setup going on.
A unit test that, although it is validly testing the object under test, can span thousands of lines and contain many many test cases. This can be an indicator that the system under tests is a God Object
Sometimes mocking can be good, and handy. But sometimes developers can lose themselves and in their effort to mock out what isn’t being tested. In this case, a unit test contains so many mocks, stubs, and/or fakes that the system under test isn’t even being tested at all, instead data returned from mocks is what is being tested.
A unit test that violates encapsulation in an effort to achieve 100% code coverage, but knows so much about what is going on in the object that any attempt to refactor will break the existing test and require any change to be reflected in the unit test.
An instance where one unit test creates data that is persisted somewhere, and another test reuses the data for its own devious purposes. If the “generator” is ran afterward, or not at all, the test using that data will outright fail.
A test case that is dependent on something specific to the development environment it was written on in order to run. The result is the test passes on development boxes, but fails when someone attempts to run it elsewhere.
A unit test which compares a complete output when it’s really only interested in small parts of it, so the test has to continually be kept in line with otherwise unimportant details. Endemic in web application testing.
A test that at first glance appears to be doing no testing due to the absence of assertions, but as they say, “the devil’s in the details.” The test is really relying on an exception to be thrown when a mishap occurs, and is expecting the testing framework to capture the exception and report it to the user as a failure.
A unit test which has lots of tests for minor (and presumably easy to test) side effects, but never tests the core desired behavior. Sometimes you may find this in database access related tests, where a method is called, then the test selects from the database and runs assertions against the result.
A unit test (or test suite) that clutters up the console with diagnostic messages, logging messages, and other miscellaneous chatter, even when tests are passing. Sometimes during test creation there was a desire to manually see output, but even though it’s no longer needed, it was left behind.
A unit test which catches exceptions and swallows the stack trace, sometimes replacing it with a less informative failure message, but sometimes even just logging (c.f. Loudmouth) and letting the test pass.
A unit test that depends on items in an unordered list appearing in the same order during assertions.
A close cousin of The Local Hero, a unit test that requires some existing data to have been populated somewhere before the test runs. If that data wasn’t populated, the test will fail and leave little indication to the developer what it wanted, or why… forcing them to dig through acres of code to find out where the data it was using was supposed to come from.
A unit test with each test case method name is only an enumeration, i.e. test1, test2, test3. As a result, the intention of the test case is unclear, and the only way to be sure is to read the test case code and pray for clarity.
A test case that doesn’t even belong in the unit test it is part of. it’s really testing a separate object, most likely an object that is used by the object under test, but the test case has gone and tested that object directly without relying on the output from the object under test making use of that object for its own behavior. Also known as TheDistantRelative[5].
A unit test that relies on a specific operation system environment to be in place in order to work. A good example would be a test case that uses the newline sequence for windows in an assertion, only to break when ran on Linux.
A test that was written pass first rather than fail first. As an unfortunate side effect, the test case happens to always pass even though the test should fail.
Rather than write a new test case method to test another feature or functionality, a new assertion rides along in an existing test case.
A combination of several patterns, particularly TheFreeRide and TheGiant, a unit test that contains only one test method which tests the entire set of functionality an object has. A common indicator is that the test method is often the same as the unit test name, and contains multiple lines of setup and assertions.
A test that, due to shared resources, can see the result data of another test, and may cause the test to fail even though the system under test is perfectly valid. This has been seen commonly in fitnesse, where the use of static member variables to hold collections aren’t properly cleaned after test execution, often popping up unexpectedly in other test runs. Also known as TheUninvitedGuests
A unit test that runs incredibly slow. When developers kick it off, they have time to go to the bathroom, grab a smoke, or worse, kick the test off before they go home at the end of the day.
Well, that wraps it up for now. I’d like it if people could vote on the top ten anti-patterns listed here so I can go about writing up more detailed descriptions, examples, symptoms, and (hopefully) refactorings to move away from the anti-pattern listed.
- Frank Carver
- Tim Ottinger
- Cory Foy
- Joakim Ohlrogge
- Kelly Anderson
This entry was posted in Test Driven Development by James Carr. Bookmark the permalink. Pingback: Fear and Loathing : Refactoring to Anti-Patterns
A unit test which sits in a test case with several others, and enjoys the same (potentially lengthy) setup process as the other tests in the test case, but then discards the some or all of the artefacts from the setup and creates its own.
One of a cluster of test case structure anti-patterns along with Stranger/Distant Relative, Mother Hen and probably Long Conversation
This is really a sort of “test case envy” and would probably benefit from refactoring into a separate test case. In more subtle cases it might also be asking for a shared base class for the two test cases.
If left unchecked, this can lead to test cases where hardly any tests make use of the common setup process, but its done for every test anyway.
One thing to avoid is the temptation to just shove all the initialization into the common setup, which just confuses and slows down the whole process. This can lead to a Mother Hen (below).
A common setup which does far more than the actual test cases need. For example creating all sorts of complex data structures populated with apparently important and unique values when the tests only assert for presence or absence of something.
This can be a sign that the setup was written before the tests themselves, which is a subtle bit of up-front design easily hidden in an otherwise fully TDD process.
One of a cluster of test case structure anti-patterns along with Stranger/Distant Relative, Cuckoo and probably Long Conversation
A unit test which, even though it initially appears simple, seems to require an ever increasing amount of the application to be created or initialised in order to make it pass.
This sort of test can take up a disproportionate amount of developer time, and lose the benefits of the fast turn-round TDD cycle. In the worst case, TDD is effectively abandoned while the all or most of the application is developed using an untested up-front process to make a single test pass.
This is common among developers new to TDD who are not yet comfortable with “faking” a response before moving the focus of design to a lower level.
A unit test which (typically because it requires non-public access to a class under test) needs to be created and run at a particular place in a package hierarchy. This initially seems perfectly reasonable, but can unexpectedly fail if the class under test is moved to a new location or (worse) split so that its behaviour goes to more than one such location.
One of a cluster of implicit assumption anti-patterns along with Local Hero, Operating System Evangelist and Hidden Dependency
It should probably be noted that this is still an active area of discussion, with many people arguing that it is a necessary or even benficial approach to design by allowing testing of non-public access.
A unit test which tests behavior no longer required in the application. Both the behaviour and the test should probably be deleted.
A test case which is so closely coupled with a particular class under test that it distracts from the need to test important interaction between classes. Commonly found in a test package structure created for Homing Pigeons where there is no obvious place to put tests for class interaction.
A test case that tests multithreaded conditions with the same thread ordering every time, or random data with the same random sample data (or seed) every time.
A unit test that requires a connection to internet with access to the outside world in order to run.
A test that creates persistent resources but doesn’t clean after itself.
Developing the code first and then repeating what the code does with expectations mocks. This makes the code drive the tests rather than the other way around. Usually leads to excessive setup and poorly named tests that are hard to see what they do.
This is closely related to James excessive setup anti-pattern and I’ve seen it with mocks. Instead of redesigning the subject under test so that it doesn’t require excessive setup; the testcase is redesigned so that it is easier to create a complex setup.
Making sure to exercise all code in order to reach the required level of X% testcoverage. But not really testing anything.
Occured when every test method makes initialization of the target. Must be refactored in the factory method.
Not a single test to confirm the system works.
A test that changes system settings.