SaaS based Test Management Software

30 days of world cup in South Africa are over now. We have watched many of the games, and followed the stories around the games, and the teams. One of the more interesting facts, now also talked about in the media, is that it was a “teams” world cup rather than a “stars” one. Seems that teams won games, and not individuals. We thought we could learn a few lessons from this, and from a few other observations we’ve made during the games.

Think of your company, project and team and you’ll see it applies nicely.

Built right

Teams that are built right, have more success. Germany, Spain, Netherlands are teams that come from countries with a tradition of working right with players, starting at a very young age. Nothing is left to luck or dependent on talent. hard work, teaching the basics and long term plans brought them to the top.

• Hard work – working with kids, taking them all the way, step by step, until they are ready and old enough to join the “real thing”. There are no compromises about training, more than once a day for many years.
• No room for luck – starting with little kids, all the training, diet instructions, psychological aspects and physical conditions are all designed and monitored by the best experts. Plans are made carefully and trainers, coaches, doctors and others are following each step making sure everything is done as it should be.
• Talent is not enough – a team can’t rely on one or even a few talents. A team has to be a team, each one contributing his part in the total effort to achieve the team goals. Building the right mixture of roles, and making sure all are in social harmony is important as much as each individual talent (and take Argentina as an example for talents vs. team work and achievement).

Work Gradually

Many teams try to skip the work and time of gradual and right process. They spend money (lots of it) on bringing the best players in an effort to win NOW. In most cases it doesn’t work. The examples are many – take Real Madrid football club and its failures to win the championship in the Spanish league, although bought all available expensive players. On the other hand, Barcelona has one of the best youth football academic clubs and they enjoy the products of it every year in the shape of 1-2 great new players (so many examples here, best known is Lionel Messi who grew in the Barcelona football club academy).

• Don’t just bring the best there are out there. Make sure your environment and culture encourage learning, professional growth and get yourself better “home players”.
• Make sure your “players” can work together. Sometimes 1+1 is more than 2 and sometimes it might be less. A good team is built slowly, making sure each of its members is the right fit, not only for his role, but in the greater picture of the team as well.

Team, Team

We’ve mentioned above already the need to work as a team. France was kicked our early because they were not a team, had many internal issues between players and between players and management. Same happened to the Netherlands in a few previous tournaments, were although had many talents and great opportunity failed to win.

Your team is a group of individuals who work together. They have to be synchronized, pull the wagon to the same direction. Each one has to know what is his part of the overall work, where can he contribute mostly and how. They must be able to help each other, overcome mistakes together and take advantage of good ideas and opportunities.

They also must trust their team leader and have confidence he leads them on the right direction. You know what happens if the coach says one thing and players do something else, each one what he thinks best.

Were would we be without Passion

It was obvious during the games that teams with less talents can try to close this gap with a passionate game. Have you see the South Korean running? Did you watch Uruguay “burning” the field or the USA squad giving it all?

Now imagine you have an A team with a few great talents, with this kind of passion….

Make sure your team is passionate about their work. Involve them, give them good example, make them believe – and they’ll bring their best act.

See you in Rio in 4 years

  

As a company, which develops tools and services for the software testing industry, it is important for us to follow up on its trends and future predictions.

We like to think that since we are part of this industry, and have a dialog with others in it (such as customers, experts, competitors and others), we know what’s currently going on and where is it going to (as much as this can be known). However, it’s always good to get some feedback on this “knowledge” from researchers, who study the industry in a more methodological and thorough way.

The most common prediction is getting more and more votes. A recent study by TechNavio Insights shows that testing – like many other software development units (and IT to this matter) – it is moving to the cloud.

Is it a bird? Is it a Plane? It’s Testing

The general term of Cloud hides behind it a few different terms related to testing. There’s the SaaS (Software as a Service), the TaaS (Testing as a Service) and the PaaS (Platform as a Service, on-demand application development platform). The new research shows that many companies, including big-size ones, understand the importance and the opportunities of the Cloud and are building their services to be available in this new form.

This goes even further when we look at new offerings (from new or existing companies) that are designed to be used as over-the-net, SaaS, Cloud applications (and you can pick your favorite term). Obviously customers are not happy to pay anymore the amounts previously paid for software, in advance, and then to add servers, people to maintain them, yearly application maintenance, upgrades, etc. Why should they, if they can subscribe rather than buy, pay for the usage only rather than commit and at the same time save all the hassle of installations.

What started with the hardware is very quickly spreading to software, and specifically to software testing. Look at all the tools available now online – test management, bug trackers, automation, mobile units virtualization, project management. You name it. So far it was the new small and mid-size vendors, but we expect the big players to join. They have no other choice. 

Need for Proof

Of course. Vendors of software testing application will have to stand to the same quality those “regular” apps have. Security has to be even tighter, and performance and availability must be superb. The challenge is huge, even before mentioning the different approach to service (including support, upgrades and on-going maintenance) that these vendors have to adopt.

Unlike installed solutions, the ease of switching to another solution provider in the Cloud will enforce vendors to take their standards to a higher level – at least those who will want to stay in the game.

It’s the golden era for customers – they will soon get better solutions, easier to manage, better services and in a lower price. The opportunities for customers are huge – solutions of TaaS will enable them to re-size their testing teams at will and reduce costs significantly. They will be able to use many testers in a short notice and for a limited time and perform much of the testing efforts faster. We believe there will be vendors offering solutions, which will be combination of community access (testers) and products (testing platforms and solutions).

Software testing has a bright future. After recognized in the past few years as an important part of the development effort, testers being part of the project teams and participating in all phases starting with the design, we will see more companies doing testing (easier and cheaper), doing more testing (testers available online and on-demand), using Cloud based solutions for the testing process. 

We’ll be there.

  

Winner takes it all – Free account for a year!

Just a short post to tell all of our blog readers about our special promotion. The world is celebrating the football World Cup event, and Testuff joins the party.

We always said that testing should be fun and what’s more fun than games :-)

Now, and during the world-cup you have a great opportunity to enjoy the games, and then go back to work enjoying Testuff.

We offer special prices while celebrating the world cup games. Starting June 11th and until July 10th, any new customer, coming from one of the 32 world cup participating countries, will receive a 10% discount on any Testuff license purchase.

And it only gets better for the winners: New customers from the winning country will get a full refund for any yearly license purchase made before July 10th.

Good Luck!

  

We often receive requests from our customers, most of which are professional testers, to work with them and show them how to use Testuff when using a specific testing methodology.

Usually after a short dialog, they find the best way for them to use Testuff. We’re quite proud that Testuff does indeed fit with many different methodologies. However, the question I always wonder about is: Why do most people (i.e. testers) think that somewhere, over the rainbow, there’s a magic methodology which flies…. In my mind, it is almost obvious that no methodology can replace common sense.

         “Common Sense Is Not So Common”, Voltaire

Keeping the testing process efficient, simple and manageable is sometimes more important than finding and adopting a methodology, which doesn’t necessarily fit in with the company, its processes, people and attitude.

This post is not intended to be philosophical and does not attempt to create yet another testing methodology. The intention is to simply try and describe my “common sense” testing approach for those who want a simple 5 step guideline.

5 Steps to follow                                      

The first two steps involve some preparation:

1. Get your requirements! no mater if you are a one-man-band or a global multinational organization, if you don’t have any requirements, you can’t test. You can even have “The site should not crash, and it should be accessible using FF and IE” as your only requirement. But if you have nothing, you may be able to do a great development work but not when it comes to testing.

2. Write a simple test script for each requirement. The test script is an instruction how to ensure the requirement is fulfilled. You can write it in a simple language: “Check that all links work”, “Check that all links have a proper title when the mouse hovers over them”. You could also use any format of step/expected result. Just write something easy to understand. I don’t have any particular advice on how to organize your test scripts. Some prefer to group them by requirement (each test under its requirement), some by the product areas, some by type (GUI, DB, etc). I prefer to put them in a place where I can find them

The third step is a mental step, just to bring you to a “testing” mode and make sure it is done correctly. Think zen.

3. Freeze the test environment. No matter if you have separate environments for development, testing, release and production or if you use only one environment for all (which is probably a bad idea). The golden rule is to freeze everything before and during testing. If someone changes your test environment whilst testing, all your effort goes down the drain and you might as well not do any testing at all. Now for the actual testing process.

There’s a little twist that will make your testing better on every cycle:

4. Start testing (and improve your scripts). Just run the Application you test and follow the test scrips you wrote in step 2. If the Application fails on a particular step of the test script, simply open a defect to be fixed later by the development team. If you test something that was not specified on the script and find a defect then (This is the continuous improvement part) add it to the script and fail the test. If you find the script irrelevant or not up-to-date then update it.

* Sub-step #2 enables you start with a very basic test script, even an empty one! and fill it in as you progress with testing.

   That’s it. All done.

Another step, to be carried out outside of the testing cycle, which will help you to get better coverage of your application is:

5. Prepare for the next testing cycle. Whilst running the application in production, you will no doubt find some defects. Don’t worry! everybody does… The key is however to avoid these defects on the next release of your application. You can make sure you improve by updating the relevant scripts for the processes in which the defect was found in the first place. This way, next time you test the application you make sure to check that the defect was fixed. It also serves well for sanity / regression testing purposes.

That’s all.

The testing world has some great methodologies, and I’m sure you can improve this “common sense” five step process. The main reason for writing this post was to encourage you to start testing your application! Instead of hiding behind “it’s too complex”, “I don’t know how to”, “We need an expert to drive our strategy” excuses, simply jump in and get started. This is what Testuff is all about.

  

Usually we leave this blog for thoughts, ideas and discussions written by different people here. However, this time we figured, why not, let’s say something about how we worked hard and finally made it…

Testuff for Mac is out!

It took longer than we first thought, and along the way we realized it won’t be as easy as it looked at the drawing table, but here we are with a version for all you Mac fans. Now you can use Testuff on your OS X and share the work with the Windows users in your office. It’s the same data, using the very same user-friendly interface, now available on both Windows and Mac.

The Mac version of Testuff test management tool comes after running the service on Windows platforms for over 3 years. It is a response to many requests by software development companies who see constant growth in Mac software and users. We all feel proud to be able to launch a monthly new version on-schedule for over two years now, with many new improvements and features. Our new Mac version, released earlier this week, is an answer to great demand from our customers and members of our community.

The Challenge

One of the main challenges developers face when working to get their app to work on different operating systems, is that in most cases it requires parallel development and in essence the creation of several separate versions, each with its own development, testing, builds, etc.  Many of you are probably familiar with these Multi-platform environment problems.

This is exactly what we were trying to avoid.

Our goal from start was to have just one code-base, which serves all different OS versions. To accomplish this we did the following:

* We used Python (a multi-platform language) and only utilize packages which work on multiple OS.

* Refactoring and packaging any part which is unique to one or more of the OS.

* We automated the build and release processes.

What’s next

We will continue to release monthly versions for both the Windows and Mac clients, working with customers to improve and enhance the system.

There’s still work (Isn’t it always the case?), and the video recorder is yet to be included, but we will continue working on it, improving it and making sure it stands to the level we are committed to.

Many thanks to all the developers and testers who worked to make it happen, and to all the users who were engaged in the RC version and helped much with good feedback and great ideas.

And now we move forward to Linux, iPhone, iPad…  

  

In the testing community, lots of companies use many different automation tools. It saves a lot of trouble going through repetitive tasks that the computer is quite capable of doing by itself. At Testuff, we don’t try to compete with those tools. They’ve been around for a while, they do a pretty good job, and trying to beat them at their own game is not what we’re after. Nevertheless, we do our best to integrate with as many automation tools as we can (as we do with bug trackers). The only problem so far is that they can’t do your dishes (yet).

I ended up playing with Selenium recently, when we created a Selenium Extension which integrates with Testuff. It got me thinking: Wouldn’t it be useful (and cool) to use those test automation tools to help me keep up with stuff on my personal life as well? How about checking my bank balance, and sending an email with the latest transactions every day? What about the mobile or home phone bill? Track electricity usage? The possibilities are endless.

Many companies nowadays offer web-based access to your own information, from banks to utility companies, mobile phone operators and so on. However, logging in every time, just to check your balance can be tedious.

I’m sure many of our customers are more experienced Selenium users than we are, but I still wanted to give a few hints about making selenium a little more tedious-task-friendly. One of the main differences in running Selenium for testing and for automating those tasks, is that when you’re testing – if a step fails – the test fails and that’s good enough. All you need is to abort the script and report it back to Testuff via our Automation API. However, for checking your online bank balance, you might want your selenium script to be a little, erm, more intelligent! It should keep trying a few times if a step fails. It should perhaps restart the whole process and try again. It could in fact ignore a few small issues and carry on. Maybe the web page didn’t load completely, but the link to the next page is available. Maybe if a link or text is not available, clicking on another link and then re-loading the page helps.

We’re using python primarily. It’s a simple and powerful language, very easy to read and write, and Selenium easily exports scripts to it. I’ll just list a couple of simple helper functions that I ended up using. They are fairly self-explanatory, but feel free to comment if you have a question.

A ‘typical’ automation script will look something like (simplified slightly):

Feel free to share any other cool ideas for using Selenium, or any other automation tool.

  

• Good service is important, great service wins customers.

• Keep it simple.

• Don’t guess. Ask your users

Then there’s the pricing model. If you were asking yourself how come it wasn’t mentioned until now, it was just because we wanted to give it the place it deserves

Should the trial be free, and ‘free’ of user’s payment details? Is it better to have a ‘money back guaranteed’ option? and how about “freemium” – give it for free but with less features, and try to make them upgrade to your premium paid version? maybe none, give them a demo and then if they want to use it they should pay?

  

Last night I’ve spent just over two hours on the phone. I don’t normally talk much. Most conversations are kept to a minimum. This call was different though. I was talking to my cable Internet support staff. I have to confess, it’s a very odd, tricky, almost elusive problem, and they were very kind and tried a few things, but to no avail. My Internet connection is still unstable and doesn’t perform the way it should. They promised to get back to me, but I wonder if they will.

The experience made me wonder: what makes (or breaks) a good customer experience? Why is it that we stick around with some companies, even when they screw up? yet are prepared to ditch a supplier whose service is pretty good, just because of a single mishandled support incident? It’s clear that a poor product or service is unlikely to make customers happy even with great support and fantastic customer service. Shoddy support however can certainly bring down even the best company providing the greatest products and services.

Support. It’s a misleading term in itself. I don’t need your support. Just get your act together and sort things out for me please. And do it quickly.

At Testuff, we use many online providers, and inevitably use their technical and customer support every now and then. We also provide support to our own customers. We pride ourselves in giving quick, efficient, courteous and straight-forward response to all our customers. We don’t always get it right of course, but we surely try hard.

So how can we be sure our support is satisfactory? And what really makes us annoyed when getting poor support from our providers? We would love to hear your thoughts, but here’s a quick list of things we noticed really make a difference:

• Timing – getting (and providing) a quick response really makes us happy. Don’t mistake a quick response with an automated reply though. We want a real response, by a human, who actually understands the problem. There’s nothing more frustrating than having an urgent issue with no response for hours or even days. We do however try to be patient when the issue isn’t really urgent, and few issues are.
• Getting it - My level of frustration really hits the roof when the support representative at the other end just don’t get it. There’s nothing wrong with asking for clarification and making sure they understand the issue, but in some cases – it seems like nobody bothers even reading the request.
• Irrelevant responses – usually accompanied by some form of a template with things I should try. We understand that companies deal with many cases, and most issues can be solved with a CTRL+ALT+DELETE. Nevertheless, getting a template response with little or no indication the request was actually read by a human being can be annoying and time-wasting.
• Plain wrong or inaccurate responses – we are all humans, and we all make mistakes, but it does appear like some companies do it more frequently and consistently than others. With some of our providers, we used to just get too many inaccuracies (and in some cases, blatant lies).
• ‘We don’t know’ – as funny as it may sound, I have much more respect to companies who don’t necessarily know what the problem is or how to resolve it. Obviously, it can’t happen too frequently, and I expect them to investigate and resolve the issue eventually. There is nothing wrong with an honest, straight-forward reply, even if they don’t know the answer.
• Being pro-active – this is subject to debate, and we don’t seem to always agree about it at Testuff. I personally like getting messages from companies telling me there’s a problem, even if I didn’t even notice it, or we really weren’t affected. This is not a pure support issue, but I think it adds to the confidence customers have in companies. If a company tells me when they’re having issues, I know it’s the kind of company who would really work hard to avoid it. Conversely, when the server is down and I’m the one who seem to break the news to the company?! I’d be out the door in a blink of an eye.
• Communication – This is partly related to being pro-active. Do you get notified when an issue is investigated? Even, or particularly so when no progress is made? As I was writing the post I noticed my Internet is performing better. Checking further I think the problem may have been resolved by my ISP. I didn’t however get a call from their support staff as promised.
• Taking ownership – Acknowledging a problem is important. Nobody likes being blamed for something they’re supposed to be doing fine, but taking ownership of problems when they take place is crucial.
• Language and Courtesy – Bad language can translate to bad manners. It is very culturally dependent however. When we reviewed some of our support responses, we realised that the way a sentence is constructed can hugely affect our customer satisfaction. This is also true when requesting support. When you get a one-line email with “X doesn’t work. Fix it.” (or similar), it’s very hard to try and be helpful.

So what’s on your list? Did we miss something out? What do you love/hate about the support we provide at Testuff? and How can we make things better?

  

• 2009 was also a year of growth for us. We have been fortunate to acquire many new customers.
• We kept on the good service and smooth operations.
  
• Not even a single customer left us because they weren’t satisfied. We had only two customers whose projects were complete and didn’t need a test management tool any more (and we already know one of them will be back in January )
• Our support team had an average response time of less than 3 hours to any incoming email.
• Our web site has been upgraded, slowly but surely, and we have received good feedback on the new home page (http://www.testuff.com/).
• On going improvements to our back-end infrastructure.

Oh yes, we didn’t manage a few things….

We couldn’t escape a few version releases with bugs, for which we had to release a new version the following few days. This is something that, as a testing tools and services provider, we should try hard to avoid