Before taking out your boat or yacht out for sailing, there’s a Pre departure checklist to follow. This list is a long one, and goes from checking your registration papers (valid and present in the boat), all the way to verifying you have all of the required emergency equipment. In between there are dozens of things to check. Such as:
- Personal flotation devices
- Sound producing devices
- Lights and shapes
- Distress Signals
- Fuel and oil
And the list goes on and on. When you get your skipper’s license, you’ll get to learn it.
Each of the items on the list, is a like a suite of tests. Tests can be:
- Ensure bilges are reasonably dry
- Ensure pumps are not running excessively
- Clean any spilled oil or waste in bilges to prevent overboard discharge
Etc. You get the point. And yes, you are right to notice that it’s very similar to software testing.
But show me one skipper that goes through the whole list before every sailing. There’s no such person, even not that OCD guy in the corner….
As a skipper, I can tell you that the items that will be checked are different from one sailing to another. They are dependent. Dependent on quite a few variables: What is the type of sailing planned (leisure vs racing), what’s the max planned distance from the docking site, who will be joining the sailing (experienced vs first-timers, grown ups vs kids), when was the last check done and how far/deep did I go then with it, how familiar am I with the yacht. And there’s more dependencies, and other parameters that would come into account and might change my decision what to check and what not to check.
Same goes for software testing.
Testing for the right reasons
You have a full test repository, with probably – at least – hundreds of tests ready and waiting to be executed. Running all of them will ensure your software/application works fine, and is clean of bugs (at least as close as it can ever get to being clean).
But this would take time. A long time, and many testers hours. Which also means money.
Is it really needed? Do we need to go and test everything again each time before we go out with a new code?
Maybe if you are planning a long sail (major version) or maybe if you have many newcomers for it (many first time users of the app, and a need to make a great first impression). Perhaps you will be sailing in rough seas (test what happens with many users, load testing), or you are not familiar enough, yet, with the boat (application is still not mature, hasn’t gone the mileage). In these cases you may be right to have a full testing cycle, working to complete all tests and making sure you are on stable ground (ha ha).
There are other scenarios, though. Is it that you will be sailing with experienced other skippers (application launched to internal team)? Or maybe it is just a one hour out-and-in of the port sail (minor version)? Maybe you sailed just the other day and went through the whole long checklist, then (just a few bug fixes since the last major version)? In such cases you may want to consider doing just sanity testing, or regression only.
If you have your automated tests in place, it is even more called for. The routine actions, the main features and functionalities are tested automatically, and you can concentrate your manual testing on only the important tests – important as set by the planned use case of the application.