Today, I wanted to talk about a topic that affects just about everyone within the software test management world. In fact, it affects a great many professions, both technical and creative.
I have writer friends who tell me that with any given document, they can edit it until it’s perfect, come back 24 hours later, and find even more things to fix. The same goes with artists, programmers, entrepreneurs – there’s always more you can do. A final brush stroke here. A new feature there. The list goes on.
At what point do you stop? When is enough… enough?
Usually in these posts, I try to provide my own insights about important software testing topics – thoughts gleaned from years of experience. In fact, we covered a very similar topic about a month ago (see: Can Software Testing Management Ever Become Excessive?) and received some pretty interesting feedback from many of our users.
I still wrestle with this concept. We all do in one way or another. So rather than pretend to have all the answers (which I don’t), I’ll simply describe the different schools of thought that I typically hear about, and then invite you to share what has and hasn’t worked for you in the past.
Software Test Management Ends When the Project Manager Says So
This is the easy answer. You put down your software testing management tools whenever your boss tells you to. He or she needs to get the product to market, end of story. Your job is done.
It’s a comforting answer – but only if you’re not the project manager.
Because I usually lead project builds at Testuff, this particular philosophy doesn’t serve me well. The decision to halt or move forward rests squarely on my shoulders. If you find yourself in the same position, then read on.
Software Testing Ends When the Product Delivers As Promised
Okay, so no one is looking over your shoulder. You’re the boss, and thus, you’re responsible for stopping or continuing the testing. When do you call it quits?
Well, many software test managers throw in the towel when the product does what it’s supposed to do. Simple enough.
At least in theory.
In actual practice, this approach is a lot more challenging than it sounds.
Testing and development are 2 sides of the same coin, but they are still 2 different sides. Programmers (i.e. developers) have a habit of adding more and more features – often at the request of the marketing department. Consequently, testers have more and more things to “test” as the scope of the original project expands (see my earlier post on how quickly projects can get out of hand).
At any given moment in time, your product’s core features might be “perfect,” but do they mesh nicely with all the add-ons and extras that continue to crop up?
In most cases, the answer is no (that’s why software testers have such secure jobs).
This makes setting a timeline very difficult. At some point, you must lock in the features and halt development. Only then will you be able to stop testing.
Software Testing Never Ends: the Importance of Incremental Improvements
Another school of thought (the one we subscribe to) is that software test management never really ends. You should “always be testing” and release incremental updates over time. This is one of the many benefits of a SaaS model. Your product is a living, breathing entity that has no beginning or end (see my earlier post on how software must constantly evolve to stay alive).
It’s a pretty solid approach – like I said, it’s what we use here at Testuff. But, it doesn’t really answer the question.
For any given update, you must still decide if and when to launch that release. Even if you do monthly updates like we do, the decision to halt or continue must be made. In fact, you’ve actually made the problem a more frequently occurring one, although the stakes tend to be proportionately lower since you’re doing updates instead of major launches.
The point is, no matter your timeframe, you must decide whether to run more tests or not, period.
No Easy Answers in the Software Test Management World
It can be maddening. On the one hand, we want the finished product to be perfect, and thus, will invest whatever resources are necessary to achieve that goal. Software testers are natural problem-solvers who love to tinker, tweak, and prod – indefinitely.
On the other hand, we’re constantly chasing a moving target. Our jobs are never finished. If we look for 3 things to fix, we WILL find 3 things to fix. And if we look for 17.5 things to fix, we’ll find 17.5 things. It’s almost as if we create more work out of thin air.
I’d be really interested in hearing about your own experience. When do you finally put your software testing management tools down and pat yourself on the back for a job well done? Please share your thoughts down below. Maybe you’ve discovered a great formula that really works for you.