Web application Automation testing -Part 2-Approach

In my previous post, we talked about the introduction to “designing a test automation framework”. In this post, I will discuss about the factors that needs to be considered before finalizing test automation frameworks.

Before stepping into phase of test automation, we must be clear on certain facts

1. We cannot really implement 100% automation in the project. There are certain types of testing which we can do more efficiently using manual testers and we don’t really need to invest our time in writing or evaluating scripts for it. Some of the examples can be Usability testing, Extensibility testing etc.

2. Manual tester should be willing to learn some OOPS concept of programming and should be at least aware about concepts of writing a program. You cannot really create efficient scripts with any paid tool also if you are not aware about concepts of programming.

3. Base of any test automation is test cases designed by manual testers. All of your functional test cases should be managed properly in order to execute proper test automation phase.

4. People who are actually participating in test automation framework designing should be clear on what actually they are expecting from a test automation framework.

5. No automation can replace manual testers’ participation in successful delivery of any product development lifecycle.

6. In some applications, some part might have designed using flex or any other platform which might not be supported by tools that I have explained in my previous post. So in order to avoid surprises in between we must be clear about which part and what level automation we are going to achieve in our project.

7. Myth about test automation is that one should not do test automation if record and playback is not available but with the tools I have listed we can design it in a way that we don’t even need record and playback however we will need so background of programming which can be built up among team with few trainings.

8. Collaboration with dev team is important because you might need to ask dev people to make changes in code in order to support some tweaks in test automation e.g. Most of the ajax calls related to dropdown value change in a webpage should be based upon change event rather than click

Above are some of the facts which we should know before starting any test automation framework designing else we might not get what we have expected!!!

Expectation from test automation framework

1. Framework should be robust, flexible and easy to maintain.

2. Framework should be designed in a way so that it can support frequent changes in the requirements

3. It should not add overhead to team for creating new test scripts and maintaining old ones else most of their time will go into modifying and fixing old scripts.

4. It should be adoptable to new tool support if we find any good tools for automation in future

5. It should support strong reporting of scripts failures for analysis

6. It should be integrated with continuous integration framework like cruise control if we have any in our project.

7. It should support data driven approach for test scripts and should have strong exception handling.

8. It should be used by dev team to create their own unit test cases for Web automation testing

9. It should handle complex scenarios like ajax calls

But is it really possible to cover all these expected behavior from a test automation framework??

Let’s see and discuss it in details using a tool Web Driver i.e. selenium 2.0…

Web application Automation testing-Part1-Introduction

Designing a Test Automation Framework

In Today’s arena, with increasing number of people using internet applications most of the product companies are coming up with web based application to fulfill the need of large customer base of internet users.

The important aspect of any website development life cycle is how much testing efforts you are putting to make your websites more stable and error free. To achieve this you cannot really rely on only manual efforts. You must need a robust test automation framework in place which will make the life of manual testers easier by doing repetitive functional testing for each nightly build so that manual tester can give more time to exploratory testing and can find more hidden bugs from the system.

With most of the companies adopting agile development methodology, the companies have to put more efforts in manual testing to find out the bugs reported into system due to changes in requirements.

Before going into web based application development most of the people are sure and clear on how they are going to develop an application? Which technology they will be using? blah blah… but what about testing this application?? People are not clear most of the times on How to test this application unless they actually land into a testing phase. They rely on manual testing and suddenly with each sprint of an agile development they start realizing that there is huge burden on manual testers with each sprint as efforts of testing an already developed and newly developed functionality increases as sprint goes. And suddenly they start feeling that we need test automation so that regular functional test cases can be automated.

But the question is HOW???

Testers are really not sure about how to go about creating a test automation framework? What factors we should consider? Which tools should we use? Should we use open source or should go about the paid tools in the market? What will be the cost of the paid tools? Can we afford it?

All these questions come to our mind and till the time we figure out how to set up a test automation framework we realize that 2 more development sprints are completed and now we are having more stuff to test and cannot really find a time to think about going automated.

Sounds familiar!!!!!! Yes this is the situation in most of the companies working in web based application development (Don’t know how much of you are agree with it?). I personally faced all these problems in web based application development project when we used to do manual testing.

Now a days, most of the companies going for test automation of web based application with open source tools like selenium, Sahi etc in order to keep the cost of product development low as much as possible. But do really open source tool can achieve the level of test automation on which we can rely on like any other paid tool taking care of the test automation framework??????

The Answer is “YES”

There are some revolutionary tools available in the market which are not only open source (FREE) tools but also those are that much stable that you can really rely on them for designing a test automation framework.

I will list down some of the tools which are my personally favorite and I have used them to create nice test automation framework.

1. Webdriver: Also called as selenium 2.0 more information can be found on website for this tool

2. WATIN: Read as (whats –IN) . “Web Automation Testing In .NET “This tool is basically designed for web based development in .net

3. WATIJ : Same as WATIN for java development. “Web Automation Testing In .JAVA “

Today most popular tool that many of the companies are using is traditional selenium. No doubt the tools is powerful and supports many of the languages for designing and writing test scripts and also provide multiple browser support. Then whats the problem with it??

Limitation of traditional Selenium as I can see

1. Not much good html finding methods for web elements

2. Use of additional selenium rc server while executing the test scripts adds overhead

3. Xpath method for finding elements, fails in cases if there are any changes in the screen or by any reason and in turns fails the test script. It takes more time to fix these errors when your test scripts reaches the count of 1000+ than actually testing the application.

4. There is no surety that test script failed is due to bug in system . It can fail even selenium rc got any exception or if element is not properly visible to proxy and we are really not sure why it is failed in short less reliable.

I will not talk about the paid tool because the most important limitation for it is cost J

The tools that I have listed above have overcome all this limitations of traditional selenium and have really revolutionized the way of test automation.

First I will compare the these 3 tools so that you can see and check it if it satisfies your automation needs

Below is the table showing some of the comparing attributes for these tools. You can play with these tools and can see more features as needed by you to design your automation framework. Please note that I really don’t know that whether these tools can be used for flex based web application because I have not used it. So if you application is designed using flex then you need to ask this to online community on extending these tools for flex UI testing Or you might need to wait till the owners add APIs for testing web application in Flex.

Features\Tools ->	Web driver	WATIN	WATIJ
Multiple browser support( Firefox,chrome,IE etc)	Yes	IE only	IE only
Record and Play back available?	Not needed	Not needed	Not needed
Language support	.NET/JAVA/Other	.NET	JAVA
Pop up handling	Not good/Will be avialable in next release	YES	YES
Exception handling	Use language support to handle this	Use language support to handle this	Use language support to handle this
data reading/ writing facility	Use language support to handle this	Use language support to handle this	Use language support to handle this
Learning curve required?	should be aware of OOPS concept and language in which we are developing a script	should be aware of OOPS concept and language in which we are developing a script	should be aware of OOPS concept and language in which we are developing a script
Reliable	YES	YES	YES
Reporting tools	External tools can be used like MbUnit/TestNG	External tools can be used like MbUnit/TestNG	External tools can be used like MbUnit/TestNG
IDE	Visual studio/Eclipse	Visual studio	Eclipse

Above are some of the features of tools and more exciting is that tool owners are adding more and more features to it with each release and believe me these 3 tools are going to be king of automation testing because they are providing more and more features same as you would get in any paid test automation tool like QTP.

This is all about tools that you can use for web automation but it is only the first step in test automation designing. What we really need before we actually start working on test automation writing.??

I will be writing more to this blog about test automation framework designing... Coming soon..