I ended up playing with Selenium recently, when we created a Selenium Extension which integrates with Testuff. It got me thinking: Wouldn’t it be useful (and cool) to use those test automation tools to help me keep up with stuff on my personal life as well? How about checking my bank balance, and sending an email with the latest transactions every day? What about the mobile or home phone bill? Track electricity usage? The possibilities are endless.

Many companies nowadays offer web-based access to your own information, from banks to utility companies, mobile phone operators and so on. However, logging in every time, just to check your balance can be tedious.

I’m sure many of our customers are more experienced Selenium users than we are, but I still wanted to give a few hints about making selenium a little more tedious-task-friendly. One of the main differences in running Selenium for testing and for automating those tasks, is that when you’re testing – if a step fails – the test fails and that’s good enough. All you need is to abort the script and report it back to Testuff via our Automation API. However, for checking your online bank balance, you might want your selenium script to be a little, erm, more intelligent! It should keep trying a few times if a step fails. It should perhaps restart the whole process and try again. It could in fact ignore a few small issues and carry on. Maybe the web page didn’t load completely, but the link to the next page is available. Maybe if a link or text is not available, clicking on another link and then re-loading the page helps.

We’re using python primarily. It’s a simple and powerful language, very easy to read and write, and Selenium easily exports scripts to it. I’ll just list a couple of simple helper functions that I ended up using. They are fairly self-explanatory, but feel free to comment if you have a question.

A ‘typical’ automation script will look something like (simplified slightly):

Feel free to share any other cool ideas for using Selenium, or any other automation tool.

We’ve seen why automated testing rocks. As I’ve stated before, it’s more accurate, work faster, harder, for a longer period of time, for less money, with no complaints and no quitting. We’ve also seen why human manual testing rocks. Humans are self maintaining, adaptive to the environment, creative, recognize objects better than machines, have built-in natural language processing, and also excellent learning abilities.

So what is the summary of it all? Fire all humans and replace them with automated testing machines? Throw all automation out the window and hire some testers for cheap from India? No!

The world doesn’t actually divide into black and white, but we quite often mistakenly think it does (a bug in our brains? The misguiding of Western society? Both? None? who knows). We don’t have to be crazy like that Israeli company who fired all testers in favor of automation, though who knows, maybe for their needs of completely repetitive well-known testing it was the right move. We also don’t have to reject any kind of automated testing at our company and stick to flesh and blood testers only. The answer is that we can enjoy both worlds, the shades of gray in between the black and the white.

Be a cyborg. Part man, part machine, it has all the advantages that humans do, but some off the hook technological enhancements that allow it to have beyond human capabilities. To take the analogy back, this means that we enjoy the best of both worlds – the automated testing world and the manual testing world and use them both to our advantage:

1. Automate boring, repetitive, high accuracy tests. Humans don’t like to do them and don’t do them as well as the automation does. By automating these kind of tests your QA will be happier since you will lift the annoying load off of their shoulders and you will get better testing results via the automation.
2. Give humans creative testing tasks. As a complimentary item to the previous one, challenge the human testers, use their natural learning skills, adaptivity and creativity to find bugs way outside the scripted tests. Don’t let them spend their time only on pre-scripted scenarios. Let them do some Exploratory Testing, new test designs, maybe even learn some programing so that they could write some useful utilities.
3. Work the automation overtime, not the humans. Having people do overtime for too long under pressure increases work stress and basically drains all the motivational juice out of them. This is why high-pressure testing, such as sanity, might be better put on the automation. However, watch out not to pressure the humans who execute the automation!
4. Use low maintenance automation. It’s a real pain in the ass when the automation requires too much maintenance in terms of time and effort. Don’t be afraid to go back to the drawing board and design your automated testing from scratch. Check why it takes such high maintenance and either solve those issues or move tests to manual execution if necessary.
5. Humans test some things better than automation. Given the current technology, computers don’t know how to do many things as well as the humans. In the testing world this is especially true about GUI. Sure, there are solutions for web testing and functional testing for simple GUIs, but if your app is more complicated than the average software Joe you will quickly see that the automation doesn’t do. If these automated tasks suck for your testers, maybe even consider off-shoring them to some Indian testers who wouldn’t mind doing some repetitive tasks for good cash.

If you still don’t believe me that you should become a cyborg, watch the trailer for Robocop and see how cyborgs improve law enforcement in crime filled future Detroit. No man and no machine would do to clean Detriot of its thugs, but only a man machine combination that is Robocop:

But wait! It is time for the humans to fight back, to show why in certain aspects their manual testing is way better than any testing automation can be. Let’s see what they got going for them that the silicon based life forms cannot beat.

Human Testing Pros

1. Self maintenance. Humans are (usually) capable of taking care of themselves. If there is some sort of system malfunction like an illness, hunger, sadness, or whatever, humans can at least attempt to solve it by themselves. The automation cannot do so. It needs someone to maintain it. If some sort of malfunction happens, human intervention must take place. Be it a network cable gone loose, a fallen web service, an air-con malfunction, the machine cannot go about taking care of itself.
2. Adaptivity. Testing automation lacks any kind of intelligence. If, for example, a connection failed to open, it won’t go about investigating what happened. A whole series of tests could fail because a network cable went loose, thus rendering hours of testing automation useless. A human on the other hand can use her wits to investigate what happened, find that loose cable, and go on with the proper testing. Also, if some change in the AUT was made the automation will probably fail. The human on the other hand will easily determine if the change is acceptable and act accordingly.
3. Creativity. Whereas the computer follows the scripted scenario religiously, people look in between the scripted lines and outside of the test altogether. They will find bugs even in things they weren’t supposed to test and be able to come up with new tests for the future. Of course, let’s remember it was the humans who programmed the testing automation to begin with, it isn’t able to design tests on its own.
4. Object recognition. Even a five year old could easily beat any computer running any algorithm when it comes to recognizing objects. Thus, and despite so many efforts made, testing automation has problems testing GUIs. A human on the other hand doesn’t really care how a UI was implemented. She can easily recognize buttons, windows, scrollbars, or anything else and get testing with it. She can see if the colors are OK, if images are displayed properly, videos are streamed in good quality, etc., things that the testing automation has big trouble doing.
5. Natural language skills. Humans may give and receive abstract instructions in natural language and act. A human tester receiving an instruction such as “test the site also works with IE7″ can easily go about doing so. The testing automation must be programmed with exact instructions in a programming language. It takes much more time to give instructions to machines and involves testing and debugging, once again by humans, to make sure the automation “understands” the instructions properly. Not to mention that automation cannot test something like a help file, that it contains good help with correct grammar. It just doesn’t understand the human talk, not out loud, and not even if we hand it in a text file.
6. Learning. Humans can learn new features, new technologies, different ways of doing things, and append new knowledge to their hard disks. Testing automation cannot do so. It needs the programmer to step in and give it exact instructions what to do. Otherwise it will never expand its knowledge base and just keep doing the same things that it knows forever, not doing anything new at all.

Once again, feel free to respond if there is something missing in the human onslaught against the testing automation. Do join me once again for a summary of the fight between human and automated QA!

Testing Automation Pros

1. Accuracy and rigorousness. Testing automation is extremely accurate and rigorous. Be it counting (try to count to 20,000 out loud and see how far you get without making mistakes or getting tired), mathematical calculations, even the computer’s memory seems much more accurate than our human memory. While a human may skip a detail here and there in a manual test, the automation doesn’t accept any funny business. An FTP connection wasn’t established? Fail the test! The exact test message didn’t appear? Fail the test! Of course as long as the test was written properly, automation is always on full concentration, never gets lazy, and doesn’t miss a thing.
   
2. Speed. The automation performs tests faster than a human. It just runs through the tests without caring about any interrupted emails or Facebook messages, easily shuffling between whatever tasks it has to do with a flick of the command line.
3. Testing 24/7. Theoretically at least, putting aside malfunctions. OK, so maybe it’s more like 20/6, but still, automation can work many more hours than humans can and at times where most humans don’t like to work, like nights, weekends and holidays. It doesn’t need any rest, no lunch breaks since it gets a constant supply of delicious electrons right from the power plug, no paid vacations. It just works as long as it doesn’t break down.
4. Cheap cheap. An automation and maybe one person to operate and maintain it is cheaper than paying for a small team of testers who would have otherwise had to do the get the same testing done manually. Not much doubt about it. Spend a few thousand bucks for some machines, feed it aircon and electricity, get it maintained by a pro, and that’s it really. So you need more machines and more people to operate the automation? That’s OK, how many testers would you have needed otherwise and how much would they have costed? A whole lot more.
5. No complaints. It doesn’t matter how dirty, repetative, boring, or time consuming the testing is. Machines ‘gladly’ go at it (even though they don’t seem to have emotions), without any need for challenges or promotions or conditions or a raise.
6. No quitting. Your star tester may leave the team in favor of a better paying job at another company. Automation won’t quit on you, at least not by choice, at least not as far as we know. While people come and go, the automation is there to stay and continue doing its job loyally. It may malfunction, then again so do people when they get ill, angry, fall in love, or whatever obstructs them from doing their job.

Did I miss something? Disagree with one of the points? Make sure to comment about it, or forever hold your silence. Do join me next week when the humans strike back after the machines’ emotionless attack as we take a look where manual testing of human QA exceeds automated QA.

Humor aside, some winds in the QA world are blowing this way, bringing forward the machine uprising, or Judgement Day if you will. I’ve recently been informed that a major company in my country fired an entire team of software testers because the automation can do their job!

I can imagine their linear line of thought. Testing automation is cheap. We don’t need to weed out thousands of dollars for monkey manual tester salaries. Instead we would only need to pay for the electricity the machine and the aircon take up, plus one person to operate it. It does the same job, maybe even better since the machine can run tests 24/7. The results could even be more accurate since the automation doesn’t miss a thing.

So what do you think about such a line of thought? There is an ongoing hype pro automation, that automation is here to replace all the manual testing that we do. Whatever repetitive action a person does when testing manually, the automation can do. Do you agree?

Having had some automation running and programming experience myself, I’m going to dive into this issue in a short series of posts. I hope you won’t be shy, and leave some good comments what you think about the whole man vs. machine issue in regards to testing, and in general if you wish as well. Stay tuned for more!