Testuff has offices based in Israel and Europe. Our customers are spread around the planet. And they develop and test software products for end users who are even more spread out.
What’s truly remarkable – the vast majority of everyone in this network has never (and will never) meet in person. Even many software development teams work remotely on opposite sides of the globe, communicating exclusively via VoIP and email.

It’s both astounding and sad at the same time. We develop SaaS-based software test management tools so you can create and test software products for end users. And 99% of everyone throughout this chain is a nameless, faceless stranger, connected only through 1s and 0s.
But we still are connected.
And with each passing year, these connections only grow stronger:

• End users demand better software products
• Testers and developers require better tools

Consequently, we work tirelessly to improve our software test management platform to ensure that this cycle continues.
Every month (for the past 50+ months), we’ve released new updates to our SaaS testing suite – and we’re on track to release 50+ more over the next 2 years.
But not all improvements are entirely software-based. For example, we recently added 2 new data centers in Japan and the US, bringing our grand total up to 8.
I know what you’re thinking – how does having 8 data centers spread across Europe, Asia, and the Americas make our SaaS test management tool “better?”
Well, for starters:

• Many of you are now geographically closer to a Testuff server than you were before. This ensures better and faster performance.
• Your software testing data are even more secure.
• You share your server with fewer customers. This not only means better performance, but it also means faster and easier disaster recovery in the unlikely event of a failure.

All of these benefits are very real – and easy to measure.

But arguably the greatest benefit of adding new data centers is that it closes some of the distance separating all of us. Chances are, we still won’t ever meet in person. But with each new addition to our global network of data centers, we inch closer and closer to becoming neighbors.
To learn more about SaaS testing or to download a 30-day trial of our software test management platform, click here.

Abstract

In today’s world, a significant portion of development projects in software engineering follow the Agile development methodology. One major disadvantage of the Agile development methodology is incorporating testing. During a sprint, a specific number of small tasks is defined (Sprint Backlog), and it is hard to determine whether the time allocated for testing will be enough to both test all the items and fix all the errors and bugs found. In the sequential software engineering life cycle methodology, the testing phase either comes directly after the development phase in a well scheduled and defined manner, or is a separate phase utilizing two different environments but still only executed after the development phase ends. A project based on Agile methodology focuses on splitting the project into iterations; each defining its own set of tasks that combine to make requirements. In many cases, the unit tests are no more than another mandatory task within an iteration. In order to avoid schedule delays and other problems, creative testing must be given a special emphasis. Adopting an Agile-based development methodology must also result in an ‘Agile’ way of thinking and planning with regard to the various testing phases. In order to meet the changes in requirements and match the flexibility of a project based on Agile methodology, planned tests must be creative in order to allow adaptation to the changes as well as the schedule.

Introduction

Software life cycle models describe the different phases of the software’s life cycle and the order in which those phases are executed [1]. There are many software life cycle methodologies and each company should adopt its own, based on the needs of each individual project. The basic sequential pattern is as follows: requirements definition, analysis, design, implementation, and testing [1]. Each phase produces deliverables required by the next phase in the life cycle [2].

After formulation and approval of the SRS (software requirements specification) documents, comes the ‘Testing Life Cycle’ [7] phase in which we determine how the system will be reviewed and tested in a way that covers most of the possibilities for a system crash.

The testing life cycle is divided into three parts:

• STP – Software Test Plan: This details the plan of action for the system tests – what we are going to test in the system and how we plan to test it (type of tests such as black box, white box, integration test, acceptance test).
• STR – Software Test Description: This contains a detailed plan of the tests including the test scenarios, input data for each scenario, execution method, and expected results.
• Final Summary Document: This contains all the test results from the testing phase, testing team recommendations regarding moving the system to the next phase, and a summary of the errors and bugs and
  their severity.

In sequential-based software development projects, system tests begin after the development ends. The tests themselves are executed by a specialized testing team belonging to either the developing company or by an external software testing company [1, 2]. This presents a big problem. Because testing starts after the development process, the developers are given feedback by the testing team close to the final project deadline and, in the case of a severe bug (as determined by each Product Owner individually per project), this can cause a major delay and/or a hasty ‘quick fix’ which reduces the quality. In many cases, delays in the development process will shorten the duration of the testing process because there was a need to meet the deadline, meaning the product will not be completely covered in terms of testing.

These problems and more [3], such as unexpected requirement changes and the need for more commitment from the team towards the project, require a change in approach. The current answer is that more and more projects are implemented using Agile methodologies. The problem with Agile methodologies is that testing is not done in a fixed pattern at the end of the development phase, but instead after each package or integration of packages [4, 5].

‘Agile development life cycle’ [6] is a term for several iterative and incremental software development methodologies and the manifesto of Agile methodologies contains four important principles:

• Individuals and interactions over processes and tools,
• Working software over comprehensive documentation,
• Customer collaboration over contract negotiation, and
• Responding to change over following a plan.

In Agile testing, there is a short feedback loop between the team members and the Product Owner which replaces sending official emails and talking to the business and developers via the Test Manager. Agile Testers are part of the cross-functional team, talk directly to developers, and have their say in all phases of the software development life cycle. With Agile testing you can influence developers to think about testability in their code and you can help understand and refine new requirements.

Testing is embedded into the definition of done in every sprint. Undone work cannot be released and classic cutting corners by skipping testing cannot happen.

Testing tasks are part of a story, the same as any other type of task, and anybody can pick up and execute a testing task. Design experts and developers need to think about the testability of the product.

Agile testing drives development by refining acceptance criteria and questioning stories during iteration planning. In Agile testing, you can use pair testing together with the developer or another tester. You can also help developers design tests or create automated tests from which both of you will benefit. One of the first things an Agile tester does is to tune the acceptance criteria and write straightforward test cases which can be used by developers to drive their development – TDD.

In the Agile development life cycle, early testing is highly recommended. Early testing means testing in phases of gathering requirements and design. Agile testing starts working on stories, evolves into test-driven development, and ends with automated acceptance testing on continuous integration for every new change committed to the source code repository.

There are several different methods for implementing Agile development in a project, such as XP (Extreme Programming), Scrum, etc. [6,7]. While each of the Agile methods is unique in its specific approach, they all share a common vision and core values. They all fundamentally incorporate iteration and provide continuous feedback to successively refine and deliver a software system. They all involve continuous planning, continuous testing, continuous integration, and other forms of continuous evolution of both the project and the software. They are all lightweight, especially compared to traditional waterfall-style processes, and inherently adaptable. What is more important about Agile methods is that they all focus on empowering people to collaborate and make decisions together quickly and effectively.

Extreme Programming is a discipline of software development based on the values of simplicity, communication, feedback, and courage. It works by bringing the whole team together in the presence of simple practices, with enough feedback to enable the team to see where they are and to tune the practices to their unique situation. Extreme programming teams use a simple form of planning and tracking to decide what to do next and to predict when any desired feature set will be delivered. The team produces the software in a series of small gully integrated releases that pass all the tests that the customer has defined. Extreme programmers work together in pairs and as a group, with simple design and obsessively tested code, improving the design continually so it is always just right for the current needs.

“Extreme Programming is obsessed with feedback, and in software development, good feedback requires good testing. Top XP teams practice ‘test-driven development’, working in very short cycles of adding a test, then making it work. Almost effortlessly, teams produce code with nearly 100 percent test coverage, which is a great step forward in most shops. (If your programmers are already doing even more sophisticated testing, more power to you. Keep it up, it can only help!) It isn’t enough to write tests: you have to run them. Here, too, Extreme Programming is extreme. These ‘programmer tests’, or ‘unit tests’ are all collected together, and every time any programmer releases any code to the repository (and pairs typically release twice a day or more), every single one of the programmer tests must run correctly. One hundred percent, all the time! This means that programmers get immediate feedback on how they’re doing. Additionally, these tests provide invaluable support as the software design is improved”. (www.xprogramming.com/what-is-extreme-programming)”

Scrum [6] is an Agile software development model based on multiple small teams working in an intensive and interdependent manner. Scrum employs real-time decision-making processes based on actual events and information. This requires well-trained and specialized teams capable of self-management, communication and decision making. The teams in the organization work together while constantly focusing on their common interests.

Based on a simple V model [9], a scrum project could have several levels of tests in every sprint [9]. In the lower level of each sprint, backlog item developers performed a unit test. On the next level up, product backlog (user stories), testers in the team performed system tests. And at the highest level, acceptance tests were performed by the customer against the project goals. After each sprint, the team additionally performed integration tests. Testers are requirements stakeholders and all of the above tests are performed against a specification. Each product backlog item is implemented and tested, and the team has two choices for the testing schedule – during the implementation of the item or at the end of each sprint.

Problems in the Agile testing process

From our experience in working with several different companies, the main problems in this development model tend to result from an inability to perform a focused series of tests at the end of the project. It is necessary to test each unit (sprint), and every time a unit is integrated into the product. Another key problem is due to multiple similar tests, as we often run tests more than once (one time per unit, multiple units), mostly during integration. It is also possible that, after a sprint and integration of a unit, the Product owner will decide to change the nature of the unit or its functionality, resulting in further repeated testing.

Because work is done in separate groups, with each group working on a different task in the Sprint, it is possible for one team to identify and solve a bug, and for another team to encounter and start to fix a similar bug, unaware that the other team has already found a solution for it and resulting in a duplication of effort.

In an ideal world, each sprint would be divided equally into a development (coding) phase and a testing phase. However, in reality, development (coding) usually takes more than half the sprint schedule, and, in addition, during a sprint it is possible to encounter problems such as not meeting the deadline and/or errors and bugs that take time to fix. This leads to a further reduction in the days designated for testing which can lead to the release of an untested (or not fully tested) unit or, at best, delaying the task to the next sprint.

Implementing Agile methodology is usually done at the start of development, where the working methods are better defined, and it is easier to implement automatic testing in such projects. The reason being that these tests are simpler and cause less difficulty in terms of communicating bugs and information between the groups, making it simpler to ‘commit’ (run) the automated test and wait for the results. As a result, the development teams incorporate automated tests into the development process, causing them to think they are covering all the needed tests. In such a scenario, the problem lies in incorporating manual tests, which are the primary source of bug detection, into the automated tests. This is due to the fact that manual tests require better communication between the development team and the testing teams, so much so that you might consider them to be a single team.
This is a tricky process, both for professional and personal reasons of the stakeholders. In addition, manual tests require the application/program to be ‘ready for release’, otherwise these tests would fail immediately. This level is hard to reach during the early phases of development and is usually attained at a later phase. Also, there is a basic methodological problem – still no ‘best practice’ method that clearly shows how to combine the manual tests into the Agile project. There are some guidelines and accumulated knowledge, but the process has a long way to go before gaining real proven experience in the field.

“Agile projects have some specific problems for testers that go beyond a lack of comprehensive documentation and the impracticality of planning much more than a couple of weeks ahead.” [10]

Possible creative solutions

Allot a set amount of time at the end of each sprint for unit testing. Should errors arise during the sprint that might push the allotted times ahead, consider dropping the current task and proceeding to the next in order to avoid ‘eating away’ your allotted testing time. Keep track of errors and bugs as well as the time it takes to solve them. Create and maintain a knowledge base based on previous knowledge to help identify possible big errors that would cause a delay in the sprint.

Avoid integration testing on small non-complex items from the sprint backlog and focus on integration tests for the whole unit. This can be made easier by focusing on test-driven development (TDD) [12], which allows our code to be ‘better’ and reduces the chance of bugs later on.

Consider spending resources on training your employees. This reduces the chances of their ‘missing’ something while testing and also expands their horizons to include different testing methods and ideas. Invest in a proper system for tracking bugs and bug fixes. This significantly reduces ‘double testing’, allowing you to keep track properly of what has been tested and fixed, and what is as yet untested.

A major factor in causing a project to fail and/or struggle is the desire to finish as much as possible in the least amount of time. Minimize the number of tasks for each delivery process and lower your expectations for each delivery. If everything is functioning well after the integration tests, add additional requirements. Emphasize the importance of each task with regard to two issues: the complexity of the task and the difficulty level of the task (Risk Factor). Based on this, we need to multiply our allotted time by the complexity factors to give us an indication of a ‘safety margin’ for each sprint. The complexity factors should be determined by each company individually, based on their previous knowledge of similar projects, their team’s level of expertise, and knowledge of the subject itself.

Make sure the requirements are broken down into the smallest possible tasks. While this creates more tests, they are smaller and allow for more extensive testing per task, reducing the chance of faulty/buggy items and, thus, reducing the likelihood of a bug during integration that causes schedule delays.

Summary

While it is impossible to create a fixed pattern for testing in an Agile project, due to the nature of Agile, if you take into consideration the possible solutions provided in this article, the chances of a critical time-consuming bug occurring during development are significantly reduced. As shown, a non-Agile project has a specific time period for
testing, usually after the development phase, which allows for easier planning and execution. Agile projects require constant changes to the project schedule (mostly due to requirement changes by the customer) and, thus, we have to adapt our testing patterns and behavior accordingly. Implementing the suggested solutions detailed here significantly reduces the probability of a module being released without testing and/or delays to a subsequent sprint.

References

1. Stephen R. Schach. Object-Oriented and Classical Software Engineering. Eighth Edition.
2. Roger S. Pressman. Software Engineering, A Practitioner’s Approach. Sixth Edition.
3. Kai Petersen, Claes Wohlin, and Dejan Baca. The Waterfall Model in Large-Scale Development
4. Dean Leffingwell. Scaling Software Agility: Best Practices for Large Enterprises
5. Victor Szalvay(2004). An Introduction to Agile Software Development
6. Ken Schwaber (2004). Agile Project Management with Scrum
7. Robert C. Martin (2002). Agile Software Development, Principles, Patterns, and Practices
8. Kit, E. (1995). Software Testing in the Real World. Wiley
9. Nabil Mohammed, Ali Munassar and A. Govardhan. A Comparison Between Five Models Of Software Engineering. IJCSI International. Journal of Computer Science Issues, Vol. 7, Issue 5, September 2010
10. David Talby, ArieKeren, Orit Hazzan, and Yael Dubinsky. Agile Software Testing in a Large-Scale Project. July/August 2006, IEEE
11. James Lyndsay (2007). Testing in an Agile Environment
12. Kent Beck. Test Driven Development: By Example

About the authors

Shlomo Mark is a researcher at the Negev Monte Carlo Research Center (NMCRC) and the Software Engineering Department, both at the Sami Shamoon College of Engineering (SCE), Be’er-Sheva, Israel.

David Ben-Yishai and Guy Shilon are research-assistants at the Negev Monte Carlo Research Center (NMCRC) and the Software Engineering Department, both at the Sami Shamoon College of Engineering (SCE), Be’er-Sheva, Israel.

This article is a collaboration between the three authors and Testuff, Ltd. (www.testuff.com), a company which develops software testing SaaS solutions.

What’s New?

• Test Editing and Planning Improved:
• Bulk Actions now supported on a lists of tests, in the Test screen. Faster and easier.
• Move tests between labs with a simple right click option.
• Simplified process of Adding Images on Steps in the Test Editor.
• Better presentation of requirements sync’d from an external tool (Supported for Jira and FogBugz) by showing the tracker requirement ID with the name.
• And More …
• tester_name field now included on the bug reporter defect template.
• Assigned-To field added to our Assembla integration.

Visit our Website for a full list of all new features in all versions since 2007.
Testuff. There’s always something to wait for.

* Take a look at the list of tips we gathered here, for your convenience.

But some updates get us super-charged. What makes this month so different?

Well, we just released even better integration for JIRA and FogBugz – 2 of the most popular commercial bug trackers on the market.

I know what you’re thinking.

“If JIRA and FogBugz are so great, why are we taking credit for this integration? Shouldn’t kudos go to their respective development teams?”

You’re absolutely right. But we’re not taking credit for the hard work that JIRA and FogBugz have put into their robust bug trackers. Rather, the enthusiasm stems from our ability to weave seamless integration into our own software test management tool.

Without such integration, we’d be faced with 3 alternatives. I’m not sure which one is worse:

1. We could have attempted to develop a whole new tracker, embedded in Testuff, competing with other trackers. We decided though, that we’ll leave this part to the experts (who, by the way, are doing a great job) and concentrate on what we are experts at – test management.
2. We could try to anticipate which bug tracker you’d want to use and weave this into our platform. The downside is that you’d be locked in and couldn’t switch to any of the other bug trackers on the market. Our experience shows that users do switch bug trackers. More than that, what if we don’t anticipate correctly?
3. We could let you choose your own bug tracker, but require that you figure out how to integrate this 3rd party technology into our test management platform. Sounds fun, doesn’t it?
   But because our goal is to incorporate hassle-free flexibility, you have many more options.

If you already use either of these bug trackers and have yet to try Testuff, making the transition to our platform will be seamless. No new learning curves. No technical workarounds. No ruffling feathers as team members are forced to reinvent familiar workflows or processes.
Testuff Version 1.53 newly enhanced features seamlessly integrate with the updated management requirements that come with JIRA and Fogbugz. This means easier testing for you, complete with better synchronization for your requirements, and as before with your defects. This also means higher quality software products for your end users.

Short disclaimer.

At Testuff, we often advise against becoming overly excited about new features – especially those in long comparison charts. So at first glance, our enthusiasm may seem hypocritical.
But we’re not against features. After all, where would test management be without them?
Instead, we advise our customers to be wary of features that don’t add value. What use are bells and whistles if you’re never going to use them or if they don’t make you a better software tester?
But we honestly feel that the new and improved integration that comes with Testuff 1.53 will make you a better tester for the reasons already listed (faster, easier, etc.). In fact, we’re sure of it since this integration was one of the most requested features from our community. You asked and we delivered.

– For complete details on everything included in this most recent release, click here.
– And if you don’t already use Testuff, be sure to take advantage of our 30-day free trial.

TEL AVIV, Israel, March 13, 2013 – The number of software testers using email for bug tracking has declined significantly in 2012, according to a new survey published today by Testuff, a provider of manual and automated on-demand software testing and defect reporting solutions.

Testuff conducts an annual survey of new bug tracker usage statistics based on a survey of global customers. The newly released data showed that the number of those using test management tools relaying on email for bug tracking has declined to 16.79 percent compared to 27% in 2011.

Testuff data shows also that testing groups are increasingly using standalone bug trackers alongside their test management solutions. At the same time, only 21% of testers use no dedicated tracker at all, with the majority of them working in smaller testing groups.

According to the findings, the most popular bug tracking solution is Atlassian’s JIRA – a tool which enjoyed rapid growth in recent months. Overall, the number of JIRA users jumped to 29% in 2012, from 13% in the previous year. Other popular tools include FogBugz (14%) and Bugzilla (6%).

“The relatively low representation of open source bug tracking tools reflects consumers’ growing willingness to pay a premium for customer support when selecting SaaS software testing solutions,” said Arik Aharoni Testuff CEO. “Testers are moving from the so called ‘free’ open source solutions, as they realize that they are actually quite expensive when hosting and installations costs are taken into account.”

Testuff conducts a yearly survey by collecting data from its global community of customers in more than 43 different countries. By offering integration with 23 separate bug trackers (including the exclusive ability to integrate multiple trackers simultaneously), Testuff continues to develop extensive knowledge of usage numbers, consumer trends, and tracking data. The company releases this free yearly review for the benefit of the software testing industry.

We discussed the switching costs of moving from your current test case management system to Testuff – and determined that the time, money and effort were minimal. But they’re not non-existent.

So are these switching costs small enough to justify the extra benefits you receive from using Testuff?

Let’s see.

Testuff Switching Benefits

• Integration & automation – Our test case management software seamlessly integrates with a range of bug trackers, automation tools, and commercial management processes. This reduces the need to find workarounds or add-ons in the future – activities that can drain both time and money, and more than that lets you keep your current project’s set of tools in place.
• Exploring new methodologies – whereas most software testing environments are locked into a select few methodologies, Testuff is adaptable. This flexibility allows you to work in familiar environments or embrace new and potentially more relevant methodologies when working on future projects.
• Increased productivity – Testuff removes much of the “grunt” work of traditional test management systems by making setup, work and analysis more intuitive. This means that you spend less time thinking about the tool and more time doing the actual work, and thinking about the results and implications. Ultimately, this leads to faster software testing, fewer errors, and easier fixes.
• Peace of mind – As this blog demonstrates, on-site storage is not as safe as previously believed. Cloud storage, with its numerous redundancies and safety checks, can be a better option – especially if you have a dedicated team of experts working around the clock to ensure the safety and integrity of your data.
• Lower price – Testuff’s SaaS-based platform is usually lower than the amortized cost of traditional on-site test management systems. Plus, there are no training, maintenance, or upgrade fees. These are technically costs (and perhaps should have gone in Part 1), but once you make the switch, these lower costs become tangible benefits. Remember that software testing accounts for 80% of development costs. Anything you can do to lower expenses is a bonus.
• Flexible licensing – When you purchase an on-site test management system, you have to anticipate the number of licenses you’ll need. If your team becomes smaller, you’re stuck with extra licenses that you’ve already paid for. But with Testuff’s SaaS testing software, you can scale the number of licenses up or down depending on your current needs. This lean approach to software testing saves you money in the long run.

How do these benefits stack up to the costs?

Well, if you ask me, the answer is easy. With greater flexibility, productivity, ease-of-use, adaptability, integration, and security, Testuff emerges as a clear winner. These benefits far outweigh the time, money, and effort required to begin using our platform.

But I’m biased. Besides, any cost-benefit analysis is a subjective exercise that ultimately depends on individual needs.

I can’t make the decision for you.

What I can do, however, is make the decision easier. You can download Testuff today and begin using it free of charge for the next 30 days. Try it out. If you don’t notice an improvement in productivity, or if you feel that it takes too long to learn, then you pay absolutely nothing.

Anytime you change products, platforms, or services, there are “switching” costs and (hopefully) “switching” benefits.

Switching costs are inevitable. It doesn’t matter if you’re changing from McDonalds to Burger King… or iPod to Android… there is a cost.

As the user, you must factor in:

• Any additional money spent on the switch
• Time invested to learn the new product or service (even if it’s just a fast food menu)
• Time invested to locate the new product or service (i.e. where is the closest Apple Store)
• The effort required to actually move over to the new product and service (ex: importing data from one platform into another)

Whereas switching costs are inevitable, the benefits are not – they’re simply implied. If you receive no additional benefit from making the change, then the switch isn’t really worth it.

More specifically, before moving from Burger King to McDonalds or one test case management software to another, you must determine whether the potential benefits outweigh the inevitable costs (time, money, and effort) of switching.

How Switching Costs and Benefits Relate to Software Testing

At Testuff, we often boast that the switching costs of our platform are minimal while the benefits are unlimited.

But what does that mean exactly?

How do you determine whether or not the time, money, and effort are worth it all?

These aren’t simply philosophical questions. If you feel that moving from your current test management platform is too much hassle – or too costly – you probably shouldn’t make the switch. But… if you discover that your team becomes more productive and/or cost-effective using Testuff, then it’s totally worth it.

In Part 1 of this post, we’ll look at the “costs” of switching to Testuff. In Part 2 , we’ll explore the benefits. And then leave you to determine if abandoning your current test platform makes good sense.

Let’s take a look at costs first:

Testuff Switching Costs

• Installing Testuff – With regards to both money and time, the installation investment is minimal. Whereas on-site servers require hefty upfront fees for hardware installation, our SaaS platform is a simple download. And as always, the first month is free, so you can try our test management system without risk.
• Training costs – Again because Testuff’s intuitive design, and UI based on Internet and Windows standards, the learning curve is small. Most users are up and running within the first hour, as opposed to after weeks and months of studying manuals or attending workshops.
• Adjusting to new workflows and processes -Testuff is flexible enough to work with any methodology. You can continue working in the same familiar environments to which you’ve already grown accustomed – only the data storage is different.
• Data transfer from the old platform – This cost ultimately depends on the system you’re currently using. But if your files are based on standard protocols and formats, our convenient uploader allows you to safely and quickly transfer your data with a few mouse clicks.

But even with minimal costs – there are still costs. Remember that all switches require some money and some time.

The question is, do the benefits of moving over to Testuff outweigh the costs outlined above?

This will be the topic of Part 2 . Tune in next time to learn how Testuff can make your job easier and faster – at a lower cost.

• Ease of use
• Versatility
• Upgrade frequency
• Add-on compatibility
• Interoperability

But today, I wanted to explore what not to look for – or more specifically – what marketing gimmicks and “promises” you should take with a grain on salt when comparing one test management system with another.
So without further ado…

Test Management Pitfall 1: Feature Comparison Charts

This one might surprise you. After all, feature comparison charts are incredibly popular and “useful” tools for comparing 2 or more related products.

But don’t be fooled.

With the right approach, a feature chart can position a Toyota Camry higher than a Rolls Royce. If the Rolls Royce doesn’t have cup holders, a detachable CD player, or seat warmers, it must be an inferior vehicle, right?

Remember that feature comparison charts are not a very accurate reflection of your needs and wants as a user. The manufacturer simply takes stock of all the bells and whistles of his product and then shows you how the competitors are missing those bells and whistles.

Test Management Pitfall 2: Lack of Specifics

In the US, if a food manufacturer adds a dash of pepper to the recipe, it can claim that its product is made from all-natural ingredients.
The same trend happens in test case management software. Don’t be impressed by generic – and often misleading – statements like “Integration with Jira.” The casual reader might believe that this platform truly integrates with Jira and offers real-time synchronization – when in fact, it’s not really offered or making the 2 work requires a lot of back-end customization (performed by the user of course).

• integration with
• synchronization with
• compatible with

… these are loosely defined ideas that vary considerably from manufacturer to manufacturer. Be sure to read between the lines.

Test Management Pitfall 3: Hidden Prices & Fees

Barbie doll marketing is a business concept in which you lure users in with a low price (i.e. the Barbie doll) and then hit them with expensive add-ons (the house, the clothes, the shoes, etc.).
You see this with modern printers as well. The machine is cheap – the cartridges are killer.
Unfortunately, test case management software suffers from the same trend. The platform is reasonably priced – in fact – a bargain. But the add-ons will get you. The developer will pretend like these add-ons are optional, but if you need true integration or recording capabilities or debugging – these add-ons are most certainly not optional for you.
Also beware of switching costs – the money you have to dole out when moving from your current platform to a new one. The software might be affordable, but with installation, on-site maintenance, upgrades (now and later), storage, technical support – the price can quickly skyrocket.

Test Management Pitfall 4: Superlatives – Best, Fastest, Most

This is another marketing favorite in which test management developers try to lure you in with hyperbolic statements about performance and features. There are 2 problems with this approach:

• Best what? Fastest what? And more important, is this a feature that truly makes a difference? Or is it just a useless column in a bloated comparison chart?
• Who is the ultimate judge of best or fastest? These are just buzzwords. Unsupported claims. After all, how many cafes legally claim to have the “best coffee in town”? They can’t all be correct, can they?

The point is, you need to decide what is “best” for you. Each project is unique. Not even Testuff can be first in class in every single category (but that doesn’t stop us from trying).

Test Management Pitfall 5: Training, Manuals, and Learning Curves

This one is a no-brainer, but I’m constantly amazed by how many testers overlook the importance of start-up times and training.
Avoid test case management systems with long learning curves. In addition to spending more time and money on training, you’ll also sacrifice quality due to higher error rates.
A more expensive platform that requires minimal training may be the cheaper option – especially when you factor in upgrades and any training you’ll have to provide to future staff members.

Base Your Purchasing Decisions on Current & Future Needs

Just so we’re clear, not all marketing claims are inherently bad. Software test management developers want to position their products in the best possible light. And they’ll use whatever tools are in their arsenal.

So when you see semi-truths or outlandish claims or poorly defined terminology, this doesn’t mean you should run for the hills. It just means that you need to do a little more research so you understand the ins and outs of that particular platform.

The above is just a partial list of things to look out for. I’d be interested in hearing some of your own lists. Feel free to comment down below.

There’s much more on this version.

What’s New?

• Requirements can be now synchronized with your tracker. This cool new feature was implemented for Jira and Fogbugz, other trackers will follow soon. With this new synchronization feature you can get requirements from the tracker to Testuff, update them automatically and add your Testuff requirement’s coverage report to the tracker.
• More on Requirements:
• New options on the right-click menu for test assignment
• New right-click menu option on Tests-in-requirement list for easier test edit
• Bug Trackers:
• We’ve added the popular GitHub to the long list of supported trackers
• OnTime integration was enhanced to include new fields, and the ability to add custom fields
• Lists improvements:
• The page-lists tables support now sort of the lists, enabling better control of the table and the data you see. Simply click the column’s title to sort the list
• Bulk actions are available for the full list, no matter the pages view if list is long
• Those little things that makes it all better:
• We’ve added indications and messages for data loading processes. This let’s you know what is the server up to and prevents wrong data presentation
• Changing labels, on multiple tests at the same time, now only adds the new assigned labels (rather than replacing the previously assigned labels)
• Labels on soft-links were separated from the labels on the originating test

Visit our Website for a full list of all new features in all versions since 2007.
Testuff. There’s always something to wait for.

* Take a look at the list of tips we gathered here, for your convenience.

“A tool is only as good as the skills of the craftsman using it.”

Or this common variation:

“A bad workman always blames his tools.”

There’s a lot of truth to this.

But time and time again, products emerge that seem to defy this logic. The iPod, for example, just works. Simple design and intuitive interface make it basically idiot-proof.

Literally anyone can use it… and use it well.

Testuff Is No iPod, But….

I don’t know if Testuff’s test management system will ever reach the global popularity of the iPod (there simply aren’t that many software testers around the world). But I’d like to believe that our platform also defies conventional logic.

True, an expert tester can accomplish more with Testuff than the novice. I don’t deny this. But I also believe that our test case management software is a powerful tool that can help novices become experts – basically escorting them through the process with minimal frustration and effort.

From the beginning, this has been one of our primary goals. To create a software testing platform that was equally applicable and relevant for all levels of users.

To accomplish this, we went out of our way to ensure that each version of Testuff had:

• A highly intuitive interface. You don’t need to search around for functions and options. Everything is always within easy reach.
• Minimal training requirements. You can be up and running on Day 1. No lengthy manuals or training workshops required.
• Ease of install. It works out of the box (or out of the “download” as it were). You don’t need expensive on-site servers or an IT technician to install hardware.
• Modular functionality. Rather than release a bloated product, we offer a range of add-ons and extras you can use if and when you want to. This a la carte approach also ensures that you have a way to implement your testing methodology. Not only that, you are able to keep using the best trackers and automation tools on the market, rather than a one-size-fits-all package that is “good enough”.

By incorporating these attributes, we’ve created a powerfully robust platform that virtually anyone can use. And not just use – but use very well with minimal effort.

This is one of the hallmarks of a great tool. Instead of focusing your attention on features and functions, you can focus on results and solutions.

It’s analogous to driving a well-made car. Your attention should be on the road – not on finding the button to turn on the lights or windshield wipers.

Testuff Won’t Make You an Expert. But It Makes the Journey Easier

Now don’t get me wrong.

Although I truly believe that Testuff’s test management system is a powerful tool for anyone who wants to continuously evolve, it is a complementary tool.

By this, I mean that you should never stop learning about new methodologies and best practices. Software testing is a process – and if you ever stop growing, you’ll never go from good to great. See our earlier post on this very topic.

But while Testuff will never replace personal initiative and hard work, it will certainly make the journey much easier. Think of it as a personal travel companion that lightens the load and clears away the underbrush as you forge ahead.

Every journey begins with a single step. Take yours today by downloading a free 1-month trial of Testuff.