I ended up playing with Selenium recently, when we created a Selenium Extension which integrates with Testuff. It got me thinking: Wouldn’t it be useful (and cool) to use those test automation tools to help me keep up with stuff on my personal life as well? How about checking my bank balance, and sending an email with the latest transactions every day? What about the mobile or home phone bill? Track electricity usage? The possibilities are endless.

Many companies nowadays offer web-based access to your own information, from banks to utility companies, mobile phone operators and so on. However, logging in every time, just to check your balance can be tedious.

I’m sure many of our customers are more experienced Selenium users than we are, but I still wanted to give a few hints about making selenium a little more tedious-task-friendly. One of the main differences in running Selenium for testing and for automating those tasks, is that when you’re testing – if a step fails – the test fails and that’s good enough. All you need is to abort the script and report it back to Testuff via our Automation API. However, for checking your online bank balance, you might want your selenium script to be a little, erm, more intelligent! It should keep trying a few times if a step fails. It should perhaps restart the whole process and try again. It could in fact ignore a few small issues and carry on. Maybe the web page didn’t load completely, but the link to the next page is available. Maybe if a link or text is not available, clicking on another link and then re-loading the page helps.

We’re using python primarily. It’s a simple and powerful language, very easy to read and write, and Selenium easily exports scripts to it. I’ll just list a couple of simple helper functions that I ended up using. They are fairly self-explanatory, but feel free to comment if you have a question.

A ‘typical’ automation script will look something like (simplified slightly):

Feel free to share any other cool ideas for using Selenium, or any other automation tool.

4 if you are really into washing your clothes as a hobby.

This applies to most of our appliances and gadgets, our mobile phones, ipods and digital cameras. We usually don’t even know what the options are; we simply use those we already know, or the most obvious ones.

Have you ever read the manual?

It’s always there. In the box. The manual.

It looks nice, written in 4 different languages, with 5 content pages and an index at the end. But what’s in it? Thicker than a Harry Potter book. No one knows…It’s a mystery as no one ever reads it. Some of us keep them in a drawer, some throw it away immediately.

And this is exactly why Testuff doesn’t come with a manual or a user guide. Not only that, we have worked hard to make it unnecessary. We strongly believe that applications which need training or a pre-sales rep to use are not good apps. Just as hardware comes in a ‘plug-and-play’ mode, so should applications. This is how Testuff comes. Simply install it and you are up-and-running in 5 minutes. It’s true, we do have an online help guide, but it’s more of an elaborated ‘FAQ’ section. Moreover, based on our user feedback, it is rarely needed or used.

What about the thousands of features a test management tool can have?

Yes, we know that there are always more features a system can have. There’s always another report, another way to navigate, a new tab, a new tick-box option or a drop-down selection box. The line between a fully featured app and an over-featured app is thin, but better stay on the right side of it. Simplicity is important and it doesn’t have to be in contradiction with giving users the full experience and answer all their needs. As Leonardo DaVinci said, “Simplicity is the ultimate sophistication”, and we’d like to keep the Testuff simplicity, which our users like and appreciate so much.

Too many features, a too complicated application isn’t a good idea. Do that and now you need training, a rep to answer usability questions and you’ll end up with users using 20% of the system’s options and features anyway.

Such an approach will also lead to a higher sale price, and a higher implementation time and cost at the other end. It doesn’t work, and especially not in the SaaS business.

More features = More customers?

Our experience shows that more features aren’t necessarily what customers are looking for. Of course, they want to see that the basics (and then some) is covered well, but they will be looking at many other aspects when making the decision if both the company and the application are good enough for them.

Reliability, competitive pricing, ease of use, security, complexity of implementation in the organization, integration to other testing tools (automation, trackers), high quality and responsive support and more.

Albert Einstein said that “Everything should be made as simple as possible, but not simpler”. Testuff does offer a fully featured package. However, there’s no need to learn how to use it, these features and options are used intuitively. You won’t find the likes of “tools –> options —> …”  buttons, but it’s all still there.

We have many debates here, long discussions about new features and new options we add to Testuff – whether they are required, will they make Testuff too complicated and less friendly. How many clicks a typical user needs to click through and whether we can make the process just a little shorter. There’s never a one only true solution. It’s more a keep-within-your-strategy thinking process.

Hopefully, we never end up with a complicated washing machine………

Unfortunately, it took us longer to release Testuff 1.1 than what we had anticipated. So, after it was finally out, we gathered round the round table and did the aftermath. This is what we found out:

• We didn’t take holidays into account. The month of September was plagued with Jewish holidays. Even though we do work quite often on non-working hours and on weekends, the holidays still slowed us down, making us go to family dinners and sleep more with lot’s of food in our stomachs.
  Conclusion – Take holidays into account for the next milestones.
• Incomplete feature design. We wanted to implement an improvement in the infrastructure. Unfortunately we found out way too late into the coding that the design had failed to take into account certain scnearios and thus the feature could not be implemented properly. The code was then reverted, coding time flushed more or less down the drain.
  Conclusion – instead of trusting the design with one man only, discuss it with more people as early as possible and include QA in the process.
• Last minute major changes. Due to some performance problems in the My Tasks feature, we made last minute major changes to fix them. This caused an array of bugs and extended the testing and fixing cycles beyond what we thought.
  Conclusion – better to wait for the next version or patch with such major changes. This way the release candidate can be stabilized faster for release.
• Lack of communication. Even though I sit in the same room with the devs, we didn’t communicate about the new version enough. Turns out there were some infrastructural changes that caused bugs, and we are going to have to release another patch to fix them.
  Conclusion – hold testing meetings and discuss changes in the new version and what should be tested in particular.
• Too many failed validations. They might not be a major time killer, but they sure are annoying and most of them can be eliminated. We discovered that some bugs were closed as fixed even though they weren’t fixed, they just didn’t reproduce for the dev. Another thing we found out is that sometimes the dev didn’t understand the bug and just closed it. The bugs had to be reopened, refixed, and retested.
  Conclusion – if a bug fails to reproduce or is incoherent, talk to QA before closing it.
• Side effect bugs. Certain bugs, even though that they themselves were fixed, caused side effect bugs that could’ve easily been fixed had they been noticed following the initial fix. These kind of bugs increased the testing cycles we needed to stabilize the version.
  Conclusion – after fixing a bug, take a minute to poke around and see if anything related went wrong.

We have learned some lessons because of this version and I hope they may be of use for you as well. Did you also run into similar situations? What other lessons have you learned from delayed releases? Share it with us in the comments or write a post about it and link back to this one.