Exploratory Testing with Testuff

How to work with Testuff when using Exploratory Testing as your methodology

There are many different approaches to software testing, with most methodologies requiring the user to develop preconceived ideas about potential defects and their causes. However, one branch of software testing offers the benefit of truly objective investigation. Whether used as a standalone methodology or in tandem with others, such Exploratory Testing allows you to uncover unknown defects as you prepare software for the marketplace.
The purpose of this article is to describe how you can use and benefit from Testuff’s test management solution for your exploratory testing.
When it comes to exploratory testing, the quality of a test management tool is measured by two main criteria:

  • Speed. The tester’s valuable time must be used and managed as efficiently as possible. You want a tool that allows users to easily and quickly start testing.
  • Documentation. The extent, ease, and depth with which tests and results can be documented. The ultimate goal centers around one’s ability to reproduce the defects found, without any additional information requests and communication between testers and developers. This includes the:
    • Ability to open a test with no content, just as a general idea or direction.
    • Ability to write the test while executing it, based on actual actions taken.
    • Test execution recording. A way to show the developers the findings/defects found in a comprehensive and visual way.

Now let’s see how you can use Testuff for your exploratory testing.

______________________________________________________________________________________________

No Time to Explain… Just Test!

Have you been in the situation when there is lack of product requirements or insufficient time for formal testing? And yet, you still need to ensure that the quality of the product is high?
Many inexperienced testers simply begin hunting out bugs without any specific strategy. With this approach, you can find a lot of defects, but there is no guarantee that you’ll test the most important functionality properly. This careless chase is often called ad hoc testing.
More skilled engineers remember that the goal of the tester isn’t to set a world record in the number of defects found. Rather, the focus should be on objectively analyzing the quality of new products – a process that requires walking in with no preconceived notions. There are many methodologies for accomplishing this, but exploratory testing is one of the most effective. Although it relies more on execution than on planning, the process adheres to clearly defined steps that allow us to more thoroughly understand the product, tests, and results – from start to finish.
Director of the Center for Software Testing Education and Research, Cem Kaner, originally coined the term “exploratory testing” in 1983. According to Kaner, the process includes the following steps:

process

Exploratory testing not only helps us learn how a given product works. It also helps us understand priorities, define requirements, and test the most critical components of that product in the first order. The process is iterative, allowing one’s understanding to grow with each subsequent cycle. This process makes it easier to isolate and test the software’s core functionality.

Software Testing Tools

Cataloging testing results across paper notes, electronic documents, and issue-trackers isn’t always convenient. For exploratory testing, we need a tool that allows us to:

  1. Document the requirements we’ve found.
  2. Add brief descriptions before expanding tests.
  3. Execute tests.
  4. Record the new results.
  5. Share these results.

Given the complexity and importance of these tasks, you need a software testing platform built around sound requirements and robust documentation. The tool must include:

  • Issue trackers
  • Screen capture functionality
  • The ability to add additional files and notes
  • Reporting capabilities (for sharing)

Furthermore, the software test management platform and issue tracker software must be seamlessly integrated.
In the past, you would need several different tools working in tandem to deliver these results. But now, there exists a unified platform that integrates all of the above. Testuff is an all-in-one solution that removes the need to rely on numerous tools, editors, trackers, and electronic files. In addition, the Testuff suite includes video recording capabilities, allowing you to send information directly to your favorite issue tracker. All in one place for your exploratory testing needs.
As an added bonus, Testuff integrates all of these exploratory testing steps into one unified screen. Let’s review.

Learn

First, we need to learn how the product works. Go through the main use cases and isolate the user roles, the objects they work with, and the workflows required. When learning the features, focus on the results (i.e. output) to better understand what input data steps are needed to produce those result.
In Testuff, keep everything you’ve learned in the Requirements section. You can add a sketch of these requirements and then expand them with additional information.

To quickly add the set of found requirements, follow these steps:

  1. Go to the Requirements tab page. 
  2. Click the three dots menu option, left of the Add requirement button. 
  3. Select Quick add many option. The Quick Add Requirements dialogue will display.
  4. Type all requirements, one per line.
  5. Click Add to save the requirements.

You can easily add additional information to any of the requirements while learning more about the product. Simply double-click any requirement in the list or right-click the requirement and select Edit.

Testuff integrates with external requirements management tools. If you already have part of the requirements stored somewhere else, you can easily import them directly into Testuff. The synchronization is bi-directional, so any requirements found and recorded in Testuff can also be sent to the external tool.
It’s worth noting that Testuff allows you to add requirements in any format you wish, whether it is a feature in the backlog, user story, use case, or even a classical IEEE record. You can also attach documents and images to the requirements.
You will need the help of someone who is intimately familiar with the product (e.g. Product Manager, Product Owner, Business Analyst, etc.) to set the testing priorities for the requirements and verify whether you understood the requirements correctly. Change the priorities accordingly in Testuff.

Design

With the basic product information clearly defined, the next step involves outlining future tests.
In exploratory testing, time is usually critical. And thus, you want to be able to design the tests as quickly as possible. Fortunately, Testuff removes the need to jump from one tab to another – all exploratory testing work can be done right in the Requirements tab page.
The first thing to do will be to create Suites, based on the requirements you have.

Select Requirement and click on the three dots menu option at its right, select the Link suites option and now click on Create Suite and link it. This will create a suite with the name of the requirement, in the Tests screen. This suite will be linked to the requirement, which means that Testuff will handle the relation between the tests in the suite and the requirement. 

Now that you have the suites, you can start creating a list of tests. This is straightforward:

  1. In the Requirements tab page, select a requirement you want to test.  
  2. Click the Three dots menu option at right of the requirement and select the Add test option. 
  3. Select the suite you want to include to the new test in Select suite dialog. 
  4. Type the name of the test, steps, and expected results. The example down below is a description for a simple “positive” test. Click Save and Close when you are done. 

For exploratory testing, we already have enough information to begin executing our tests. We can only make educated guesses about the tests and expected results at this moment. We’ll validate these data in future steps.

Execute

Once the initial tests are created and linked with the requirements, we’re able to run the test and learn more about our assumptions. We want it to be easy, and fast. Testuff does exactly that for you – run, record with video, report defects and comment.

In the Requirements tab, select the Tests you want to run.

Our goal for now is to verify how the product works and record the test steps. The execution protocol is important for keeping the “explored” knowledge and interpret the results. We might also find some defects during this stage.
It is recommended that you use Testuff’s video recorder to easily log everything for future reference.

Hover the test that you want to run and click the Run icon. 

The Test runner window will open. 

Now initiate the video recorder to capture all of your actions while testing:

Click icon1 or drag the icon2 icon, and trace out the desired screen capture area. Click icon3 to start the recorder. Test your application while Testuff captures all clicks, mouse movements, new screens, and messages with the video recorder tool. Everything is carefully documented so you don’t need to worry about keeping any logs.

 

The recorded video, which is now linked to the test, is essential to the exploratory testing process. It allows you to easily reproduce the steps taken during the test – without overlooking any of the details. In addition, the video also allows you to analyze whether the tests and results match up correctly.

Writing and re-writing of tests, while running them, is an essential capability of a testing tool when doing Exploratory Testing. It’s a Must have feature. You may Edit test to further describe test steps with their expected results.

Interpret

Now that you have enough information, you can create the final test cases for further regression testing:

Play back the recorded video to identify test steps and expected results by clicking icon5.

The video player not only shows the captured video, but also cursor trails, mouse clicks, input data, and keystrokes.

If you have found any defects while executing the test, select the select  near the specific step of the test to tie the bug report to that exact step.

Type the bug report description in the Bug Reporter. Captured video is already available in your computer.

Keep in mind that any tests you’ve created and defects you’ve found are only assumptions until they’ve been confirmed. Refer to someone who knows more about the product to help you understand whether your test cases, steps, and expected results are accurate.

Complete this process on those requirements with the highest priorities. And make sure the tests for these requirements are confirmed and executed. You are now ready to do another round of the Learn, Design, Execute, Interpret cycle. Repeat this cycle to obtain more knowledge about the product, and test these new assumptions as needed.

process2

When you have a set of ready test cases, you can delegate the test execution to other team members.
In the Requirements tab page, select a requirement and click the three dots more actions menu option, select the Assign test toand designate the desired tester from within the list.

In this way, the exploratory testing process easily scales up, allowing multiple team members to add their own input.

Additional Notes

  • Watch out for the Pesticide Paradox – a common situation in which repeated testing of the same defects often results in a decreased ability to isolate said defects. In order to ensure the effectiveness of the tests being performed, you must frequently revisit and rewrite the tests. Otherwise you risk glossing over unexposed code and missing out on some of the core functionality of the product.
  • Also, focus your attention on defect clustering per the Pareto Principle. In general, 80% of defects are caused by 20% of modules. So if you find a disproportionate number of defects somewhere in the product, it makes sense to add more specific tests to verify this functionality further. Fortunately, Testuff allows you to easily expand both requirements and tests, without sacrificing traceability between the two.
  • Working from the Requirements screen is a great approach, however you can get the same results while using other screens as well, mainly the Tests screen.
  • Here’s a “cheat sheet” that can help you remember each step of the Testuff exploratory testing process. It covers the entire exploratory testing lifecycle, from isolating requirements all the way up to reporting defects.

process-summary