Is Web Unit Testing A Waste Of Time?

I first heard about unit testing as a government requirement for certain applications. They ask a simple question: Do you employ unit testing (Yes/No)? There was a minimum amount of code, just enough to check “yes” and move on. I don’t think it even did anything.

My next encounter was when I created my first MVC Application, exploring the framework. It starts you out with a small site and an optional unit test application. A great way to get the ball rolling since that’s probably the biggest hurdle. I started hearing more about Test Driven Development (TDD) – the notion of starting a project by writing lots of failing test to outline your requirements, then writing just enough code to make those tests pass. Sounds intriguing, so I began reading “The Art of Unit Testing”, and experimented with Selenium, a javascript based browser plugin that basically records your actions on a webpage and plays them back for you. A very cool program, with a lot of time saving potential I could see.

But alas, feature requirements, demos and client requests always take front seat, and poor little unit test has to take a seat at the back of the bus next to documentation. It’s the equivalent of buying a bunch of fancy exercise equipment that shamefully becomes coat racks.

Time constraints aside, I have another problem, known in the TDD world as “blank slate syndrome”. I need to write a unit test that will succeed today, but would cause errors if I make changes. Not just any changes, but changes that a) wouldn’t throw compile errors, b) have any possibility of actually happening, and c)would take more time to catch and fix than writing the test and keeping the test in sync with the application. This is a very, very hard thing to do. The examples in this book, as well as the Selenium documentation, are often very short and sometimes silly scenarios like this:

homepagetitle =getTitleOfHomePage();
Assert.AreEqual( homepagetitle, “Welcome to my homepage”);

Are you kidding?

Furthermore, preparing code for unit testing means completely separating out your business and datalayers, abstracting your data access points into interfaces (good practice, but time consuming and complex), setting up a mock repository with fake data, using an inversion of control setup to switch out the real data access with the mock access in your tests, ensuring all this stays in sync as your app evolves so you don’t get red herring tests, and also writing unit tests that don’t themselves contain bugs. Alternately, do you write unit tests for your unit tests? If you’ve seen Inception, I think you know where that can lead.

All these things pose the question: is it all a waste of time?

One golden rule put forth by the author of “The Art of Unit Testing” is this: The time for unit testing is when the pain of not unit testing outweighs the pain of unit testing. But how can you measure that pain if you don’t know it? I’ve come up with a list of factors:

  1. Consequence of Failure.
    What happens if your production application has a failure? If you are developing a photo gallery for your company and someone can’t upload a picture, they send you a bug report and you fix it. If you’re programming for NASA, your bug report may come in the form of flaming wreckage. Web applications have the advantage of being able to continually fix problems without any effort by the user to download patches. But still certain web apps are vital to work, and failure may cause security leaks or lost data. Very bad for banks.
  2. Language and Framework.
    If you are working in a static language like C#, the compiler will catch the vast majority of your errors. Also, if you are using a framework, built in features are fairly stable and have most likely already been unit tested by the framework developers. Dynamic languages will make assumptions rather than throwing exceptions. Misspelled variable names, poorly casted variables or calls to non-existent functions might go unnoticed until runtime. In this case, the unit test has more value.
  3. Project Definition
    How well defined is your project? Are the business rules set in stone? Are you working in an agile or waterfall based environment? The better your spec, the easier it is to write unit tests. The tests can actually become part of the project definition in certain cases. When the client says, “this page needs a login button”, you can write a test for that. But if a client isn’t quite sure what they want and the project requirements are constantly changing, writing a unit test can be like playing a board game with a four year old who’s constantly changing the rules.
  4. Importance of Security
    Unit tests can rapidly try to insert long or malicious strings into every input on your site. Buffer overflows, SQL injection, cross site scripting attacks, brute force password attacks and the like are all great things to write unit tests against. No matter how diligent a tester is, these are tasks a computer is much more equipped to execute over and over, so failure can be found early.
  5. Expected Lifespan and Evolution
    No rocket science here. The longer you expect your project to be used, maintained, and expanded, the greater the importance of unit tests for your core features.

 

All this is great, but I still have blank slate syndrome. I’m working in a strongly typed language, with a comprehensive framework, in an agile environment where any requirement can change. Security is important although, and we do expect a long lifespan with lots of expansion. We have a framework using IoC and Mock, so we’re almost ready to go. I just need to take that first step. I’d be very interested to hear of any unit tests that have been true life savers or time savers.

This entry was posted in asp.net, mvc, unit testing, work. Bookmark the permalink.

3 Responses to Is Web Unit Testing A Waste Of Time?

  1. Willy says:

    This is a great podcast that discusses BDD as a way of getting over blank slate syndrome.

    http://www.dotnetrocks.com/default.aspx?showNum=628

  2. John Pencola says:

    @Steve:
    I’ve had similar thoughts, I don’t believe unit testing is applicable across the board. Here are some observations:

    I’ve seen unit-testing work well in medium to large teams where there are lots of integration points between applications. For smaller teams and projects, personally I don’t buy the hype to do it for the sake of writing tests.

    Its also important to note that TDD comes in many forms, some of which have nothing to do with writing unit-tests. I practice TDD, but I may not have a test suite per se.

    I’ll add a #6:
    Continuous Integration (CI). Again, this comes into play more in mid-large teams where there are multiple dependencies that need to be regression tested prior to deployment. Automated testing plays a vital role when doing integration tests.

    -JP

  3. mindstorm says:

    @Willy:
    I think I first heard the phrase “blank slate syndrome” from that podcast. They helped me term what I was doing, but the problem still persisted. I partially wrote this post to clarify in my own mind some entry points.

    @JP:
    I agree that integration points are a good place to start, making sure the left hand and right hand are in sync, so to speak. With CI, the compiler is still enough of a safety net that bug causing code changes are usually caught. One good web failure point that a compiler can’t catch is links that expect certain URL variables in the query. I have had cases where one developer will change the signature of a URL method, but forget to update every link to that method.

    On a similar note, AJAX methods normally pass in loosely typed parameters and often receive JSON or XML object, which are loosely typed as well but require certain values to work. Especially since JS is case-sensitive but VB isn’t, this is an area where I seem to have a lot of trouble with features breaking over time. Both entry points could potentially be unit tested. Selenium is also helpful in this situation too for automating testing. I’ve only used the Firefox plugin though. I haven’t gone full throttle with the RC but they claim to be able to integrate with NUnit to launch and test on multiple browsers. But it takes a lot of work and time to set up.

    I think what it comes down to is corporations need a dedicated test team, not just button clicking interns but people who know these tools inside and out, and whose goal is to find weaknesses in applications that they did not write.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>