on-demand software test management

Last night I’ve spent just over two hours on the phone. I don’t normally talk much. Most conversations are kept to a minimum. This call was different though. I was talking to my cable Internet support staff. I have to confess, it’s a very odd, tricky, almost elusive problem, and they were very kind and tried a few things, but to no avail. My Internet connection is still unstable and doesn’t perform the way it should. They promised to get back to me, but I wonder if they will.

The experience made me wonder: what makes (or breaks) a good customer experience? Why is it that we stick around with some companies, even when they screw up? yet are prepared to ditch a supplier whose service is pretty good, just because of a single mishandled support incident? It’s clear that a poor product or service is unlikely to make customers happy even with great support and fantastic customer service. Shoddy support however can certainly bring down even the best company providing the greatest products and services.

Support. It’s a misleading term in itself. I don’t need your support. Just get your act together and sort things out for me please. And do it quickly.

At Testuff, we use many online providers, and inevitably use their technical and customer support every now and then. We also provide support to our own customers. We pride ourselves in giving quick, efficient, courteous and straight-forward response to all our customers. We don’t always get it right of course, but we surely try hard.

So how can we be sure our support is satisfactory? And what really makes us annoyed when getting poor support from our providers? We would love to hear your thoughts, but here’s a quick list of things we noticed really make a difference:

• Timing – getting (and providing) a quick response really makes us happy. Don’t mistake a quick response with an automated reply though. We want a real response, by a human, who actually understands the problem. There’s nothing more frustrating than having an urgent issue with no response for hours or even days. We do however try to be patient when the issue isn’t really urgent, and few issues are.
• Getting it - My level of frustration really hits the roof when the support representative at the other end just don’t get it. There’s nothing wrong with asking for clarification and making sure they understand the issue, but in some cases – it seems like nobody bothers even reading the request.
• Irrelevant responses – usually accompanied by some form of a template with things I should try. We understand that companies deal with many cases, and most issues can be solved with a CTRL+ALT+DELETE. Nevertheless, getting a template response with little or no indication the request was actually read by a human being can be annoying and time-wasting.
• Plain wrong or inaccurate responses – we are all humans, and we all make mistakes, but it does appear like some companies do it more frequently and consistently than others. With some of our providers, we used to just get too many inaccuracies (and in some cases, blatant lies).
• ‘We don’t know’ – as funny as it may sound, I have much more respect to companies who don’t necessarily know what the problem is or how to resolve it. Obviously, it can’t happen too frequently, and I expect them to investigate and resolve the issue eventually. There is nothing wrong with an honest, straight-forward reply, even if they don’t know the answer.
• Being pro-active – this is subject to debate, and we don’t seem to always agree about it at Testuff. I personally like getting messages from companies telling me there’s a problem, even if I didn’t even notice it, or we really weren’t affected. This is not a pure support issue, but I think it adds to the confidence customers have in companies. If a company tells me when they’re having issues, I know it’s the kind of company who would really work hard to avoid it. Conversely, when the server is down and I’m the one who seem to break the news to the company?! I’d be out the door in a blink of an eye.
• Communication – This is partly related to being pro-active. Do you get notified when an issue is investigated? Even, or particularly so when no progress is made? As I was writing the post I noticed my Internet is performing better. Checking further I think the problem may have been resolved by my ISP. I didn’t however get a call from their support staff as promised.
• Taking ownership – Acknowledging a problem is important. Nobody likes being blamed for something they’re supposed to be doing fine, but taking ownership of problems when they take place is crucial.
• Language and Courtesy – Bad language can translate to bad manners. It is very culturally dependent however. When we reviewed some of our support responses, we realised that the way a sentence is constructed can hugely affect our customer satisfaction. This is also true when requesting support. When you get a one-line email with “X doesn’t work. Fix it.” (or similar), it’s very hard to try and be helpful.

So what’s on your list? Did we miss something out? What do you love/hate about the support we provide at Testuff? and How can we make things better?

  

• 2009 was also a year of growth for us. We have been fortunate to acquire many new customers.
• We kept on the good service and smooth operations.
  
• Not even a single customer left us because they weren’t satisfied. We had only two customers whose projects were complete and didn’t need a test management tool any more (and we already know one of them will be back in January )
• Our support team had an average response time of less than 3 hours to any incoming email.
• Our web site has been upgraded, slowly but surely, and we have received good feedback on the new home page (http://www.testuff.com/).
• On going improvements to our back-end infrastructure.

Oh yes, we didn’t manage a few things….

We couldn’t escape a few version releases with bugs, for which we had to release a new version the following few days. This is something that, as a testing tools and services provider, we should try hard to avoid

  

Social Media

Social media is everywhere these days. We all hear about Facebook, MySpace, Flickr, Twitter and other such sites every day. We get emails referring us to them, new add-ons, plugins and widgets are born every minute and the phenomena is covered regularly even by the traditional media. And now it’s no longer used only by individuals, as it was in the first phase. Businesses are finding it important, sometimes crucial to “be there”. Take us for example – we have a Twitter account (http://twitter.com/Testuff), we participate in testing related forums, we have a LinkedIn company’s identity and we have a blog (have you noticed ? ). Our experience shows it helps and can benefit any company, however it should be regarded as one of the many marketing tools a company has. It started with newspaper ads or articles, then came the Internet and now you have many new and exciting channels to reach your audience.

Can Social Media Tools work with “real” apps?

We were wondering if it will be possible to harness the power and appeal of these tools for other applications, business apps, those you find when you turn on your PC at work, not at home… Our intention was to find what we called “the third floor”. First floor being the individual using social media sites and tools for personal use. On the second floor corporates enter the game and use it as a marketing, sales and PR channels. On the third, our floor, we intended to use ideas and tools that were created by and for the social media, and incorporate those into how we work. We enjoy the fact that users know them, got used to them and are happy with them (100’s of millions of users can’t be wrong). It’s only a natural progression to bring those elements into the workspace. The main problem was to find a way to get more of it, than just the “social” part. In a business-like application you want to see features that help your do your job, make it easier for you to perform whatever it is you are tasked with. Purely keeping in touch with colleagues can’t help much and won’t be enough. We also wanted to keep it simple. This is the way we do things here at Testuff anyway. We thought it would be best if we start with a simple, focused idea, and take it from there.

Twitter

After many discussions and a few weeks down the line a decision was made. Integrating Twitter into Testuff. Remembering our intention to create useful features, and not just “cool” or “fun” toys, we came up with “Twistuff” (http://www.testuff.com/help/twistuff). With Twistuff, our users can follow up on their testing progress and tests results automatically, by receiving test-tweets automatically from Testuff. The user is free to choose which Twitter account to use, and they can select the type of updates as well. For a tester, a group of testers and a test team manager this can be a productivity tool, for follow up, to know about other testers test results, to get messages about identified defects etc. Not just Testuff users can enjoy Twistuff. Others in the company (developers, managers) can see the daily summaries and keep up to date too. All they need to do is follow one twitter account.

If it goes well, and the feedback is positive, we will continue and enhance it, looking for more ideas how to use existing, familiar ways to our users, to make their job more productive, and a little more fun.

  

 You probably don’t even know, and I bet you use maybe 2-3 of them.

4 if you are really into washing your clothes as a hobby.

This applies to most of our appliances and gadgets, our mobile phones, ipods and digital cameras. We usually don’t even know what the options are; we simply use those we already know, or the most obvious ones.

Have you ever read the manual?

It’s always there. In the box. The manual.

It looks nice, written in 4 different languages, with 5 content pages and an index at the end. But what’s in it? Thicker than a Harry Potter book. No one knows…It’s a mystery as no one ever reads it. Some of us keep them in a drawer, some throw it away immediately.

And this is exactly why Testuff doesn’t come with a manual or a user guide. Not only that, we have worked hard to make it unnecessary. We strongly believe that applications which need training or a pre-sales rep to use are not good apps. Just as hardware comes in a ‘plug-and-play’ mode, so should applications. This is how Testuff comes. Simply install it and you are up-and-running in 5 minutes. It’s true, we do have an online help guide, but it’s more of an elaborated ‘FAQ’ section. Moreover, based on our user feedback, it is rarely needed or used.

What about the thousands of features a test management tool can have?

Yes, we know that there are always more features a system can have. There’s always another report, another way to navigate, a new tab, a new tick-box option or a drop-down selection box. The line between a fully featured app and an over-featured app is thin, but better stay on the right side of it. Simplicity is important and it doesn’t have to be in contradiction with giving users the full experience and answer all their needs. As Leonardo DaVinci said, “Simplicity is the ultimate sophistication”, and we’d like to keep the Testuff simplicity, which our users like and appreciate so much.

Too many features, a too complicated application isn’t a good idea. Do that and now you need training, a rep to answer usability questions and you’ll end up with users using 20% of the system’s options and features anyway.

Such an approach will also lead to a higher sale price, and a higher implementation time and cost at the other end. It doesn’t work, and especially not in the SaaS business.

More features = More customers?

Our experience shows that more features aren’t necessarily what customers are looking for. Of course, they want to see that the basics (and then some) is covered well, but they will be looking at many other aspects when making the decision if both the company and the application are good enough for them.

Reliability, competitive pricing, ease of use, security, complexity of implementation in the organization, integration to other testing tools (automation, trackers), high quality and responsive support and more.

Albert Einstein said that “Everything should be made as simple as possible, but not simpler”. Testuff does offer a fully featured package. However, there’s no need to learn how to use it, these features and options are used intuitively. You won’t find the likes of “tools –> options —> …”  buttons, but it’s all still there.

We have many debates here, long discussions about new features and new options we add to Testuff – whether they are required, will they make Testuff too complicated and less friendly. How many clicks a typical user needs to click through and whether we can make the process just a little shorter. There’s never a one only true solution. It’s more a keep-within-your-strategy thinking process.

Hopefully, we never end up with a complicated washing machine………

  

In one of my previous positions (we’re talking in decades…) I had a manager who really liked reports. He was all over us with requests for reports, most of them statistics about performance vs. goals but there were others as well.

We had to create nice, fancy reports (charts and graphs), with many colors and lots of numbers on them to satisfy him and found ourselves not doing what we should have been doing – working.

Of course all of these reports were at best glanced at and then thrown to the place all papers end their life.

The fact was that there were always too many reports created, too many different views of the same data, by too many people. In reality only a few reports were used and benefited the company.
Since then many things have changed, but managers didn’t. They still want reports. Only now creating reports out of the data stored in the various applications we all have in our work environment is easy. In fact it’s usually so easy that even our managers can do it! A click of a mouse.

A good reporting system gives the option to create the reports needed, with current data and a nice simple presentation. I’m referring here to ‘reports’ mostly in their ‘chart’ form, and it is essential to remember that a chart is simply a representation of data in a visual way.

The main thing about reports is to be able to get those that have a contribution to the organization, by having the specific data, in that certain way and for that period of time. The report/chart should give such information that we’ll be able to use it to learn and improve (a process, an employee, a method, goals – something).

Its important not to create reports that no one can understand, and are overload with data and information which misses the best part of a good report (being visually simple and quick to understand).

As Groucho Marx once said: “Why a four-year-old child could understand this report. Run out and find me a four-year-old child. I can’t make head nor tail out of it”…

There’s no need, in my opinion, to have a “monster” reporting system. 97% of it will be never used. Let’s concentrate on the 3%, make it easy to use, nice to look at and flexible enough to keep all users happy.

What should a good report include and be:

• Easy & Quick to create
• Current updated data
• ‘Active’ (show figures on mouse move etc.)
• Type (i.e. line, bar, radar) relevant to the data
• Efficient display of the data
• Meaningful, descriptive title
• Axes labels
• Axes scaled proportionally
• Legend, if needed
• Gridline, where required

    We are now designing and building the first reports on Testuff. As always it is based on requests and recommendations we have gathered from many users for some time now, and hopefully by that will be what they really need and want.

Following our own way of doing things, we plan to ‘start at the start’ and grow from there.
We’ll keep you posted with the progress we make.

  

On August 6th we have reached our first 1,000 registered companies (and we posted it at Testuff is 1000 companies old!). A few weeks ago we had the bells ringing again, when our 2,000th company registered to the service. Simple math, which even we can do without mistakes, shows a nice average of 125 registrations a month in this period. In fact, the current average is actually higher and looks like the Internet famous growth potential for small businesses such as ourselves proves right, so far.

Now what?

This is all nice indeed and a good start, but surely not enough. Heard about Facebook ? Of course you have, and so have over 200,000,000 registered Facebook users (amazing number, isn’t it?). Bottom line? Facebook has yet to earn its first profit Dollar. And don’t get us wrong, we love what they are doing, respect their success and sure they’ll find the right model soon.

As a new and small business we are constantly in search of the best approach between two options – get as many registrations, become big(ger) and only then deal with generating income somehow OR grow slowly, adding only paying customers while trying to keep the expenses as low as we can, all that in the old fashioned way of actually making money from each of those customers.

It is a common understanding by now, that the “eye balls”, “registered users” and other such Internet terms are not enough any more for a business to thrive. There has to be logic, a business model, leading to a very simple result where the expenses are less than the income and the company earns money. Our goal is not “to grow” but rather “to grow smart”, meaning that growing by itself doesn’t say much about your business.

As Warren Buffet said about his basic business rules: “Rule No.1: Never lose money. Rule No.2: Never forget rule No.1.”

It should work better for you as well – we guess you’d rather get your testing services from a profitable company and not a losing (and fading?) one.

So our challenge now is to continue building good products, giving excellent service and growing in number of customers while maintaining these business rules and helping ourselves to stick around for many more years. No less important is that we’ll continue to enjoy what we’re doing, which helps us work better and hopefully makes our users happier.
We’ll check this when we get to 3,000 and share the results with you

Note: Facebook had their first 100,000,000 registered users celebrated in a huge party. They decided not to celebrate the new milestone number, as they probably realized it is better to celebrate earnings growth…

  

“Built to Last“, the successful book by Jim Collins and Jerry Porras, tells us about great organizations, real visionary companies that have prospered over long periods of time, and shows the unique practices and habits that these companies adopted. But I wonder if any of these companies’ CEO or founders could answer a simple question: “Can you promise that your company will survive?”

Particularly in this financial climate, we see more or more companies going bust, some of which are major brands. Only a few months ago they were all showing fantastic profits and no-one would ever imagine such a collapse. So how do you know who to put your faith in? What assurances can you expect to get from your business partners, suppliers and service providers?

Customers are asking us different questions, amongst which:
- How long has the company been in existence?
- How many employees do you have?
- How many clients do you have?
- What industry sectors do these clients operate in?

I guess that all these questions mean: “Are you a real company that is built to last?”, or – perhaps the real question is – are you just two students working in a warehouse with no penny, no future and no ability to survive more than a few months .

I believe that no company could promise such a thing, but what every CEO can promise is a commitment to manage the company in a responsible way and to take all the necessary steps to keep the company profitable and strong.

Sharing some business indicators that show that the company is ‘on the ball’ usually helps.

So what are our success indicators so far?:
- We have more then 1700 registered companies, with clear increase in new companies per day ratio.
- We see a steady growth in revenue over the very few months of commercial operation.
- We are earning more, and not losing money – our income is higher than our operational expenses.
- We have started expanding our business using selected affiliates.
- Our dev team releases a new major version every 4-6 weeks as planned.
- We keep meeting customer expectations and deliver feature requests consistently.
- Our support team responds to most queries within a few hours.
- We keep going.
- We keep going.
- We keep going

Sure. We can use some fancy slogans, promise that we will always be #1 and publish a press release for every new customer. But this is not our way here at Testuff.

  

This post is a personal post. so if you don’t like it, just scroll your mouse wheel to the next post in your RSS reader.

Ido is leaving Testuff and I would like to thank him for all the wonderful
work he did. He joined us when our product was in beta and is leaving when it’s very stable and in production and he had quite a bit to do with that.

During development, there is always the dilemma of whether to let developers test the product or to hire testers to do the job. I always believed that it was better to hire a tester, but if you needed some more testers, you could use developers to temporarily extent the testing team. I thought that testing was an easier job then was programming, and it made sense to me that if the developer has some free time on his hands, the company can use him as a tester as well.

After a few months with Ido I realized that I was way off. Asking a developer do a tester’s job is like asking a hen to fly – he can do it, but usually rather badly. Developers have a particular state of mind that helps them write new code, solve problems in old code and improvise solutions when the direct approach doesn’t seem to work. They subconsciously trod lightly when using the software and cautiously walk around problematic areas and places they know might now work correctly. A tester’s state of mind is altogether different – he’s always looking for new and exciting ways to bring the program to its knees and is constantly on the lookout for the smallest inconsistencies, controls that are off by even one pixel and ways the application can be improved. I’ve never thought of it this way until I had a chance to work alongside a world-class tester.

“Eating our own dog food” usually means using the tool that sell. In Testuff, Ido was eating most of the dog food while the rest of us were, well, making it. Ido was the first to use Testuff in our own testing process, and forced us to improve it to fit to his insanely high standards of user
interface and usability. He brought the same high standards to our customer support, web site and blog, which I think, is self evident.

Ido needs these days some time off after the busy months he had, planing to think and decide about his future plan calmly.

I wish him the best of luck.

Arik