<< Edmug's 1 Year Anniversary Party | Home | Calgary Code Camp >>

Writing unit tests

Yesterday I was bored.  When you take this bloke out of the Igloo and it's to warm to his tongue stuck to a parking meter he tends to write manifestos.  This one stemmed from having to fix a test fixture that failed when the tests were run in random sequence.  The result was an email outlining what I think are the primary concerns each developer should have when they write their unit tests.  These were my thoughts (expanded a bit because sometimes I care to share more with you than my coworkers).

  1. Never create module level object variables in your test fixtures so that they can be reused in many tests.  Often this will cause false positives and negatives simply due to test execution sequence.  Creating new objects in every test is preferable as it allows you to know that the starting state of those objects is clean and pure.  There can be exceptions to this rule such as creating one MockRepository (RhinoMocks) that is used to create mock objects across all tests.  If you're going to break this rule you'd better be able to justify it to me without breaking any of the other rules listed here.
  2. Treat tests as a specification of what the code under test should be doing.  If you read a test without reading the code, you should be able to say, with confidence, how the code under test is expected to perform.  I'm a firm believer that refactoring tests decreases your ability to adhere to this principle.
  3. Don't refactor test code to the same extent that you refactor application code.  Test code is utilitarian by nature and also requires high levels of readability (see point #2) and maintainability.  Because of this test code should not be exceptionally, or even remotely, elegant.
  4. If you find that you're struggling to write tests that cover all expected behaviours of the code, or to get a test to pass, then the code under test is probably too complicated.  Complex tests are an indication of complex code, which in turn should be an immediate sign that the code under test needs to be refactored to adhere to Single Responsibility Principle.  When I do code reviews, complex tests fully qualify as a code smell.
  5. Never write tests that rely on other tests to be executed either before or after that test.  A unit test should be an encapsulated piece of work that is ignorant of all external functionality and sequencing with the exception of the method that it is testing and the Setup and Teardown methods.
  6. Avoid mixing process (RhinoMocks.Expect.Call(...)) and state (NUnit.Assert.AreEqual(...)) based tests in the same test method.  This relates back to readability and documenting the expectations of the code under test(see point #2).  If you write one test for process based testing, I know exactly how to expect the code under test to behave with regards to it's dependencies.  If you write another test for state based testing, I know exactly how to expect the test to work with regards to its data.  Mix the two together and I have a muddled concoction of data and dependency information presented to me and it takes more effort to separate the two and clearly understand each.

Of these 6 points, number 6 has proven to be the most controversial at work.  I will readily admit that it is more rooted in personal preference than the other 5, but I think that I'm backing up my personal preference with some decent reasoning.  If anyone has anything to add (more points, better/different explanations) I'm all ears.

posted @ Wednesday, April 11, 2007 4:19 PM

Print

Comments on this entry:

# re: Writing unit tests

Left by Tom Opgenorth at 4/11/2007 5:42 PM
Gravatar
#4 is good too, and I've found it to be the most useful to help people learn how to break down their classes into more manageable chunks. #5 is, IMHO, the most important one and seems to be the hardest one to get people to understand. More than once I've rolled onto a project and discovered these elaborate "testing frameworks" that run tests in a certain order. As I always try to explain it: a unit test should never fail because another unit test failed or didn't run. #6 I think should be more of a guideline/rule of thumb if you ask me. Sometimes the most pragmatic way to test process is to examine the state.

# Interesting finding - 04/11/2007 &laquo; Another .NET Blog

Left by Interesting finding - 04/11/2007 « Another .NET Blog at 4/11/2007 9:52 PM
Gravatar
PingBack from http://liangwu.wordpress.com/2007/04/11/interesting-finding-04112007/

# re: Writing unit tests

Left by Travis Laborde at 4/12/2007 5:40 AM
Gravatar
#5 is a bit hard for me to grasp. I see the points, but..... In typical CRUD tests for example, I would have a test to INSERT a record. Since the test is "hard coded" it inserts a record with a specific set of data attributes. If I try to insert that same record again, I get a UNIQUE constraint exception. Thats a test too of course :) At any rate, since I have to test the DELETE method, in my mind thats the perfect reason to have the DELETE test run after the INSERT test. And it deletes the record that I just inserted. So that I can run my tests again and again without getting foreign key constraint issues, etc. An alternative I suppose would be to combine the INSERT and DELETE tests into one test. So, is that a better practice than sequencing the tests? Or is there another, better way, that I'm missing altogether?

# Interesting Finds: April 12, 2007

Left by Jason Haley at 4/12/2007 6:47 AM
Gravatar

# re: Writing unit tests

Left by The Igloo Coder at 4/12/2007 7:42 AM
Gravatar
Tom: I think that #'s 4 & 5 are the two that I use as teaching points more than any other. The interesting thing is that they don't usually get used for teaching good test writing, but rather teaching good code design. Travis: I think that if you're testing your CRUD functionality as you describe you are precariously tied to sequencing. What you need to do is break the reliance on data generated in a different test than is being executed. Don't fear though, there is a way to do this. What you should be looking at is a tool like NDbUnit which will allow you to create a known data state prior to each test run. NDbUnit will Create, Update or Delete data in the database for you prior to, or after the test being run, allowing the test (a delete for example) to be run with out relying on any other tests for data setup. A second alternative is to roll your own data setup suite (essentially duplicating the NDbUnit tool), but these usually become quite ugly to maintain.

# re: Writing unit tests

Left by Tom Opgenorth at 4/12/2007 11:58 AM
Gravatar
Travis: It Would Be Bad to combine both your update and delete into one test. If the test fails, why did it fail? was it because the update didn't work, or the delete? You can't tell. In my experience it's better to keep the tests simple, and a quick running as possible. Tools like NDbUnit aren't without their own perils (maintenance of the datasets being the biggest one I see), but it is better than nothing. Another trick that I've heard about is to use an in memory database for testing your CRUD functionality. Probably not to practical in many circumstances.

# re: Writing unit tests

Left by The Igloo Coder at 4/12/2007 1:11 PM
Gravatar
Another option is to use a testing tool like MbUnit which has built in functionality to handle test by test transactability. This allows you to rollback the database changes implemented by a test. Like Tom said though, this may or may not work well in your situations. If you're working with a data gateway that doesn't subscribe to transactions (like some of us have to) then you're not going to see any benefit.

# Interesting finding - 04/12/2007 &laquo; Another .NET Blog

Left by Interesting finding - 04/12/2007 « Another .NET Blog at 4/12/2007 8:20 PM
Gravatar
PingBack from http://liangwu.wordpress.com/2007/04/12/interesting-finding-04122007/

# re: Writing unit tests

Left by Heiko Hatzfeld at 4/13/2007 2:15 AM
Gravatar
Hello... If you want to test your crud code, then I would do so in a seperate class. Inside this class i would define a setup, that will insert the record in question into the db. and in the teardown of the test, i would remove the record (via Transactions or by deleting by hand) This allows you to test the crud in a very simple way... Just create an "empty" test and call it InsertObjectIntoDB this test will do nothing but insert and rollback afterwards. then you can create a "fetch" test that will only do a fetch vs the DB after that you can create a delete and update test to test the rest of the functions

# re: Writing unit tests

Left by Travis Laborde at 4/13/2007 5:00 AM
Gravatar
Thanks for the responses! It seems to me that each response has it's own set of drawbacks, bringing them to the point where sequences don't seem so bad after all :) I do have another reason to want *out* of sequences... Zanebug! Apparently, it ignores them. But it's a cool tool, and I'd like to use it more often. But ignoring sequences is way bad. First one test fails (since it was run out of sequence, the delete has no record to delete). Then if you run it again, two tests fail - because the insert is trying to insert a duplicate record. Oy!

# My case against weak-typed languages

Left by Coding in an Igloo at 4/22/2007 7:44 AM
Gravatar
Before you read this post in it's entirety, you need to head over&nbsp;to Justice Grays blog and see...

Your comment:



 (will not be displayed)


 
 
 
Please add 7 and 6 and type the answer here:
 

Live Comment Preview: