Sunday, July 11, 2010

Unit Testing Redux

I've mentioned unit testing before. I've extolled its virtues before. I'm not going to go over those items today. Instead, I'm going to go over how it saved my project at work.

Back in March, I proposed a new idea at work. My idea, when put into production, would actually make money for the company. I don't mean it would save money, either. I mean that we would be able to actually properly bill customers when they go over their allocations/what they're paying for. Right now, this is, at best, a difficult process. As a result, it is frequently ignored. With this project I'm building, it would now be able to become how we do business. My project will actually make money for the company, not just save it.

Unfortunately, I screwed up in my early testing and coding. I created a situation where an M:N join table was not properly being joined, because I missed a test case. The results were looking correct for the test cases I had, so I assumed things were okay. As it turns out, they weren't.

The system is in QA right now, and someone found a bug and reported it to me. After digging in, I found the problem, and realized I had missed this test case. My initial reaction was bad. How could I ever check this? How could I possibly prove that this bug was fixed, the fix didn't introduce new ones, and that the system was in good shape?

The answer comes from my already extensive test suite. I added some more test data to it, and exposed the bug. Test cases were now failing, the way they should have. I then fixed the join condition, and ran my test cases. I had some new failures, but that was because I failed to account for the new test data in my reporting test cases. I accounted for the new data in those test cases, and now everything came up green. Everything was working as expected.

Now, how bad was this problem? Without those test cases, I would be unable to be sure that my fix actually fixed the problems. I would be unable to be sure that bugs didn't exist elsewhere, waiting to be detected. I would not know that the problem was resolved. I would believe it, but I would not know it, not for sure. And with that level of uncertainty, my management might have either killed the project, or made me start over.

With those test cases, it became a line item in a status report: Bug in join condition reported, found, fixed. Tests now look for this problem, and all tests now pass.

I don't think I'm exaggerating when I say that unit tests saved my project. People laugh when I tell them I spend as much time on my tests and documentation as I do on my code. Now, I'll just point them to this story. I am so glad I found out about unit tests right now.

No comments: