Testing Frameworks in 5 Minutes

What is Testing Framework?

Testing Framework is a set of principles like: coding standards, processes and practices, hierarchies, reporting methods, test data injections etc. which should be followed during automation testing in order to get beneficial results. Test automation can still be done without a framework but, if used, produces great advantages: Code reusability, ease of scripting, reduced script maintenance cost, understandability, less manual intervention, maximum coverage, high portability, easy reporting and recovery. So, a Test Automation Framework provides benefits which helps in development, execution and reporting of automation test scripts efficiently. A Framework also proves useful in a team so, that each member follows same approach for development. A framework is not dependent on type of application (Technology or architecture).

Popular Types of Frameworks:

	Module Based Testing	Library Architecture Testing Framework	Data Driven Testing Framework	Keyword Driven Testing Framework	Hybrid Testing Framework	Behavior Driven Development Framework
Definition	Module based Testing Framework derives from OOPs concept of Abstraction. Under this whole application under test is broken down into number of modules. So, each module will have its separate test script. All smaller test scripts are integrated to build larger test script. The idea is to divide modules in such a way that application changes won't affect larger test scripts. So, scripts are easily maintainable	The common functions (based on common steps) from each script are segregated and a common library is created and functions can be called within scripts. E.g. Start and Stop a service in each test	Here Test data is segregated from script logic. Test data is stored externally in different files e.g. xml, csv etc or ODBC repositories. Test data is mapped to different sets of input and expected output values	Here Test data + common set of code is segregated into external file. The set of code is known as keyword. Keywords+Test data is stored in tabular form so, this framework is also called Table Driven Framework	It is a combination of more than 1 frameworks e.g Keywords and data used together in same sheet	Here functional validations are put in easily understandable format so, that anyone in the project hierarchy can understand it e.g. cucumber, Jbehave etc.
Advantages	Easier and cost effective maintenance of test scripts, scalable and can bear changes in application easily	Easier and cost effective maintenance of test scripts, reusability	It covers all combinations of test data, changes in data are easily accommodated, flexible and easy maintenance	Programming language knowledge is not needed here. Same keyword can be used multiple times	Advantages of all frameworks used	Programming language knowledge not needed
Disadvantages	Test data manipulations are required every time	Complicated and tedious to handle multiple sets of test data	Complex as data needs to be fed, requires knowledge of programming language	Complex and requires keyword creation knowledge

Test Automation Framework Design

It requires a well planned approach based on common, successfully implemented industry standards. A good Design should be:

• Application-independent
• Scalable and maintainable
• Should separate the testers from the complexities
• Identify and have common library for functions
• Separated test data from the test scripts
• Scripts should be able to run on their own with minimal errors
• Version control

Here’s an overview of steps to Design a Framework

1.       Identify Testing Scope

2.       Types of Testing required in project

3.       Scope of automation identified

4.       Automation Tool Decided

5.       Data Input Store:

Object Mapping needs to be done and syntax decided for Object Identifiers e.g. Login button with an alias name

All the possible test scenarios identified and test cases decided based on sequence of flows e.g. TC1 Start service and login, TC2 performs transaction etc

List of custom messages decided for error handling

Driver-Files containing list of data files, transaction ids etc

File formats identified

6.       Develop Framework:

Scripting language decided based on tool

Utility functions created based on design

In addition Driver and Worker scripts are created

Approaches identified for reusability utilities e.g. data driven or keyword driven

7.       Populate Input data store using manual or automated methods

8.       Configuration of Schedulers, so that scripts can be initiated by anyone available