Software Testing Management and the Eternal Principle of Common Sense

Posted on Oct 8, 2012 in blogTestingTips

Ralph Waldo Emerson once wrote, “As to methods there may be a million and then some, but principles are few. The man who grasps principles can successfully select his own methods. The man who tries methods, ignoring principles, is sure to have trouble.”

In the world of software testing management, this general axiom definitely holds true.

At Testuff, we constantly receive questions from new customers about the “best methodology” to apply when using our software QA testing tools. Because we’ve designed our SaaS testing suite to handle a range of established methodologies, we can quickly recommend any number of approaches for any number of tests.

But I’m always a bit amused by the question.

On the surface, such inquiries make perfect sense – of course you want to know how to use a given tool properly. But the underlying premise of “best methodology” is somewhat flawed. There are a million and one approaches to software testing management (which is why continuous training is important). But how best to approach software testing ultimately depends on the answers you seek and why?

In this, I’ve always found “common sense” to be the best guide. There are often times when no single methodology is ideal because you’re exploring a set of questions and requirements that, as of now, have never been asked.

In fact, it is from these new questions and requirements” that “new methodologies” eventually evolve. What place would Agile, Exploratory or Waterfall testing have in a world where all current methodologies are already sufficient?

In a previous post, we explored some of the more common sense approaches to software testing management, outlining 5 key principles. I could have easily expanded on these 5 and come up with a top 10. But I decided to go easy on you and condense them into a more manageable list of 3 core principles.

Software Testing Management #1: Start with the End Goal in Mind

Lewis Carroll recounts the following in his classic, Alice’s Adventures in Wonderland:

Alice: Would you tell me, please, which way I ought to go from here?
Cheshire Cat: That depends a good deal on where you want to get to.
Alice: I don’t much care where.
Cheshire Cat: Then it doesn’t matter which way you go.

It is imperative that you understand your desired outcomes and known requirements. Without these, no software testing methodology will be of any use.

  • Are you testing to make sure your software works on a particular platform (i.e. Firefox vs. IE)?
  • Do you want to minimize failed login attempts?
  • Want to ensure that your mapping software syncs well with satellite technology (see our earlier post on this topic)?

If you start too vague, no methodology will save you. Make sure you’re asking the right questions and have a clear understanding of what your software QA testing tools need to accomplish.

Software Testing Management #2: Target, Segment, and Save

However many questions and requirements you have, there should be an equal or greater number of individual scripts to test these requirements.

  • If you have 3 questions, there should be at least 3 separate scripts.
  • If you have 15 requirements, there should be at least 15 scripts.

This seems obvious – so obvious in fact that very few methodologies make specific mention of this basic principle. But you’d be surprised how overlooked the concept is. Software never deliberately deceives – after all, it has no conscience. But it can certainly lie by “omission.” So make sure you have an answer to every question.

After creating each script, it is important that you implement 2 timesaving strategies (you’ll thank yourself later):

Strategy 1: Organize your scripts accordingly. It doesn’t matter if you group them thematically, chronologically, or by product area. Just so long as the organization makes sense to you and your needs. Don’t become slave to the needs of any given methodology’s original creator. Remember, he had his own set of requirements and priorities during the creation process.

Strategy 2: Freeze your testing environment and save all work. Make sure you lock in your changes. Software testing management is a process, but each script and accompanying report is a “snapshot” of what you’ve accomplished.

Changes here and there can quickly contaminate your work and make future tests irrelevant. This is especially true in team environments that have numerous cooks adding to the pot.

I can imagine that a huge chunk of unnecessary software development costs come from failure to properly document incremental improvements.

Software Testing Management #3: Always Be Testing

Software is never truly finished. And thus, software testing management is never truly finished. And thus, you should “always be testing” – both before the product’s release and after.

Before the Release

  • If a test fails, update the script.
  • If untested bugs occur, write a new script to isolate them.
  • If all tests pass with flying colors (especially at first) be suspicious and think outside the box.

Despite the very tangible and real costs of poor product development, I find it best to approach software testing management like a game. Basically, assume that the original developer has challenged you to prove that his or her masterpiece is anything but perfect.

Individual methodologies can help you get started (yes – they’re clearly important), but it never hurts to become creative and look for ways that the software could be wrong. For this, you may occasionally have to stray from the established canon of known methodologies.

Fortunately, our software QA testing tools were designed for both the beaten path and the ones less traveled.

After the Release
Like I said, software is never finished. Even when the product is already on the shelf, your job is far from done.

  • Any bugs that may have slipped through need to be scripted and tested for future releases (it happens to all of us).
  • Monitor feature requests and customer feedback. By tapping into the collective wisdom of the crowd, you can aid your development team with future changes and compatibility issues.

Software Testing Management: Methodology Has Its Place, But….

Established methodologies are extremely important. They represent best practices that have been tried and tested repeatedly by thousands of testers out there.

But.

If you pick your favorite methodology in the world, I can pick a time (in recent memory) when that methodology simply didn’t exist. New approaches are invented all the time, and their emergence simply reflects the needs of the inventors at that particular moment.

If your needs just happen to be the same as the original inventor’s, so be it. Consider yourself lucky. But more often than not, you’ll be asking slightly different questions, and thus, need to run slightly different tests. Use established methodologies as a starting point, but use common sense as your compass.