Thursday, August 6, 2009

Negative Testing

Disclaimer: I realize my statements are generalized, and might be completely off base, but they match my experience and even if they are wrong, illustrate the critical need for a mindset required for writing good unit tests.

In their most recent podcast, Joel Spolsky and Jeff Atwood broach the subject of software testers, and how they address a different concern in software development than unit testing. The fundamental thought is that software developers chip away at a feature until it works; testers chip away at a feature until it breaks.

The problem with most unit tests written by software developers is that the goal only verifies that things work under ideal circumstances:
  • All the CRUD operations work!
  • All the business rules are enforced!

Most unit tests typically fall down because the haven't anticipated all the problems that will occur in more realistic scenarios. For example, when was the last time you've seen a unit test that verifies the length of text fields? As Joel and Jeff mentioned, even if you have some negative testing, have you verified your code works with Unicode? In other languages?

I've been guilty of validating input, but still missing the mark. Several years ago, I was working on an application that was using regular expressions to insure that integers were being entered by the user for a given field. In the first round of testing by QA, one of the bugs reported was that I was accepting negative values when only positive values were valid.

The hard part for most developers, including myself, will be in changing gears from getting our code to work to making it break. Unfortunately, the only way I know how to make this come easier it to take a lot of pride in your work, learn from your mistakes, and do your best to avoid repeating the same mistake twice.

Finally, while I don't want to spend a lot of time on the value of QA, I would like to at least echo Joel's expressed value in having a human making sure your code works. In addition to functional validation, real live people will also provide usability, performance (sometimes), and general workflow feedback, which is invaluable to creating a great user experience for your users. Think of them as someone asynchronously pair programming with you rather than being at odds with them.

No comments:

Post a Comment