I first heard about unit testing as a government requirement for certain applications. They ask a simple question: Do you employ unit testing (Yes/No)? There was a minimum amount of code, just enough to check “yes” and move on. I don’t think it even did anything.
But alas, feature requirements, demos and client requests always take front seat, and poor little unit test has to take a seat at the back of the bus next to documentation. It’s the equivalent of buying a bunch of fancy exercise equipment that shamefully becomes coat racks.
Time constraints aside, I have another problem, known in the TDD world as “blank slate syndrome”. I need to write a unit test that will succeed today, but would cause errors if I make changes. Not just any changes, but changes that a) wouldn’t throw compile errors, b) have any possibility of actually happening, and c)would take more time to catch and fix than writing the test and keeping the test in sync with the application. This is a very, very hard thing to do. The examples in this book, as well as the Selenium documentation, are often very short and sometimes silly scenarios like this:
Assert.AreEqual( homepagetitle, “Welcome to my homepage”);
Are you kidding?
Furthermore, preparing code for unit testing means completely separating out your business and datalayers, abstracting your data access points into interfaces (good practice, but time consuming and complex), setting up a mock repository with fake data, using an inversion of control setup to switch out the real data access with the mock access in your tests, ensuring all this stays in sync as your app evolves so you don’t get red herring tests, and also writing unit tests that don’t themselves contain bugs. Alternately, do you write unit tests for your unit tests? If you’ve seen Inception, I think you know where that can lead.
All these things pose the question: is it all a waste of time?
One golden rule put forth by the author of “The Art of Unit Testing” is this: The time for unit testing is when the pain of not unit testing outweighs the pain of unit testing. But how can you measure that pain if you don’t know it? I’ve come up with a list of factors:
- Consequence of Failure.
What happens if your production application has a failure? If you are developing a photo gallery for your company and someone can’t upload a picture, they send you a bug report and you fix it. If you’re programming for NASA, your bug report may come in the form of flaming wreckage. Web applications have the advantage of being able to continually fix problems without any effort by the user to download patches. But still certain web apps are vital to work, and failure may cause security leaks or lost data. Very bad for banks.
- Language and Framework.
If you are working in a static language like C#, the compiler will catch the vast majority of your errors. Also, if you are using a framework, built in features are fairly stable and have most likely already been unit tested by the framework developers. Dynamic languages will make assumptions rather than throwing exceptions. Misspelled variable names, poorly casted variables or calls to non-existent functions might go unnoticed until runtime. In this case, the unit test has more value.
- Project Definition
How well defined is your project? Are the business rules set in stone? Are you working in an agile or waterfall based environment? The better your spec, the easier it is to write unit tests. The tests can actually become part of the project definition in certain cases. When the client says, “this page needs a login button”, you can write a test for that. But if a client isn’t quite sure what they want and the project requirements are constantly changing, writing a unit test can be like playing a board game with a four year old who’s constantly changing the rules.
- Importance of Security
Unit tests can rapidly try to insert long or malicious strings into every input on your site. Buffer overflows, SQL injection, cross site scripting attacks, brute force password attacks and the like are all great things to write unit tests against. No matter how diligent a tester is, these are tasks a computer is much more equipped to execute over and over, so failure can be found early.
- Expected Lifespan and Evolution
No rocket science here. The longer you expect your project to be used, maintained, and expanded, the greater the importance of unit tests for your core features.
All this is great, but I still have blank slate syndrome. I’m working in a strongly typed language, with a comprehensive framework, in an agile environment where any requirement can change. Security is important although, and we do expect a long lifespan with lots of expansion. We have a framework using IoC and Mock, so we’re almost ready to go. I just need to take that first step. I’d be very interested to hear of any unit tests that have been true life savers or time savers.