TMV (Too Much Validation)

I normally begin each post with an anecdote explaining my motivation behind writing. In this case I was asked to add code to a user profile page to validate a zip code. The request was that I not only check the length but that I compare the zip code against a registry of all possible zip codes to see if that particular zip code exists. My response was simply “Why are we bothering?” After all, I thought, what’s next? Submit their address to Google maps to make sure it exists? Check their name against MVA or voting records to make sure they exist?

Validation is the act of verifying that data is constrained to certain rules. The rule could simply be required input, a certain format such as date, or a complex system of logic that combines constraints on several input fields. When a user submits data, a certain amount of validation is helpful. We’ve all been saved at least once by having to type in our passwords twice when they don’t match.

The question I pose is this: is there a such thing as too much validation? And if so, when does that occur?

To answer that question, we have to take a step back and look at what’s happening. Data submission of any kind is essentially a conversation between two parties: the submitter provides information and the receiver provides feedback. This could be through a web form, a paper form, or an interview. An interview requires the most time and effort by the interviewing party but offers the highest degree of validation since they can converse freely. Paper forms remove the interviewer and thus streamline the process, but provide no validation. Invalid data is submitted, caught while being processed, and the form must be fixed or returned for resubmission. Valid data must still be processed and stored by an administrator of some sort. This system has worked fine for centuries and still does, but now we have the web, which brings me full circle.

Web forms can provide instant feedback and force a user to fix errors through validation. This completely cuts out the interviewer and the administrator. It also saves time for the user during entry and shortens the processing time, which benefits everyone. The only cost to the enterprise is that they now need programmers, database administrators and the technical baggage to keep the system running. Still, this is generally a fraction of the cost for the provided return.

Validation is useful for streamlining a conversation for both parties. But once validation starts to hinder the conversation, it’s usefulness declines.

The most obvious occurrence is when validation throws false complaints. This can occur when the rules change but the code logic becomes out of date. Some examples might be the 9 digit zip code or new email formats (I once forgot to allow for European email formats). These are the simple fixes. The gray area concerns the things I mentioned at the beginning.

The issue is relevant to me not only because I code lots of forms with massive amounts of validation checks, but also because I am currently co-authoring a jQuery plugin to precede user actions with a confirmation dialog to force the user to verify their intent before committing the action. Like validation, a confirmation prompt can protect users from making mistakes, but it could also become a nuisance if it is overused. Since this is a plugin, it will be up to the consumers to decide when and how often to implement it, but to make it as useful as possible, we want the default behavior to mimic the majority of use cases.

And determining default behavior requires a bit of prediction at this phase. Hopefully, once we go live, we will get some useful feedback from the community and see how close our predictions are.

This entry was posted in html, work. Bookmark the permalink.

2 Responses to TMV (Too Much Validation)

  1. John Pencola says:

    Hey there!

    Just thought I’d drop a note to say how concise and well thought out this post was. What I come away with after reading this is more of an affirmation that we, as software developers and engineers, must always think about how, what and why we go about collecting data during that “interview” process. To your points, maybe it would be fun to follow up with some strategies that you’ve used or observed in the past for making that process easier on both ends?
    Keep up the great work,

  2. mindstorm says:

    Probably the most important thing a developer can do is to observe a non-technical user test their app. When you’ve been coding and testing a feature for weeks, you lose your objectivity. I’m often surprised by what features users will overlook and where confusion can occur.

    Other factors to consider might include how often a user will perform certain actions, how motivated is the user to complete the process and what are the consequences of “bad data”.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>