Mercury Interactive's WinRunner®
is a very versatile product that can be used with extreme effectiveness in automating Unit
and System Test Plans. Like many automated test-tools, it has the ability to record input
and "play back" a series of actions performed by the user. While this method is
a useful technique to enhance productivity in the development of scripts, it is not viable
as a long-term strategy for development nor can it effectively accommodate the
requirements of System Testing:
 |
The
recorded input can be incorrect either due to tester error during the recording session,
or the test script can be incorrect in the first place. A reliance on recording alone
would then require the entire input to be re-recorded. |
|
 |
The
resulting script contains multiple commands and statements for every action taken, is
extremely lengthy and difficult to interpret and maintain. |
|
 |
The
data contained within these scripts which is used for input and verification is
essentially "hard coded". If data must be changed (e.g. account numbers, etc.),
all scripts must be updated with the new data. |
|
 |
The
GUI file (a mapping mechanism that tracks on screen objects) generated as the user records
may be inaccurate for other scripts. For example, if the "attached text" is used
as a field (object) tracking attribute, this can be taken from data which has been input
into the preceding field. This is especially true when testing Legacy (Mainframe)
applications via Terminal Emulation. |
|
 |
Recording
of test scripts requires that the application be fully developed, and error-free. |
|
 |
Without
the addition of some programmatic intelligence, WinRunner® can only verify data
that changed since the prior run. It does not verify the expected results as
delineated in the Functional Specifications or System Test Plan documentation. |
|
 |
If
errors occur, such as unexpected screens, the recorded test script will fail, and
therefore requires someone to constantly monitor the test run in order to keep it going. |
|
What is actually required, is the development of customized test scripts
using the Test Script Language (TSL) in the development environment provided
by WinRunner®. This gives the user an interface which allows the
development of re-usable test scripts, and more control of the testing process.
WinRunner's development environment provides the foundation for
developing effective test automation. It uses TSL, a "C-like" programming
language, which must be learned by those writing the test scripts for which some degree of
previous programming experience is required. This provides the building blocks for test
automation. However, moving from "building blocks" to an automated testing
process requires a significant investment of effort beyond
acquisition of a single tool. A critical component of successful test automation is a
methodology, which lets the user produce test scripts which are robust and error free.
The Solution: The ToolKit for WinRunner®
The ToolKit for
WinRunner® is basically composed of User-Defined Functions and Utility scripts which
give the tester more control of the WinRunner® test tool. These functions and utilities
facilitate the tester's ability to use the tool in whatever method or manner is dictated
by specific testing requirements, rather than being subjected to the inherent limitations
of the test tool.
Our Automated Testing
methodologies employ a structured approach to testing which makes use of the ToolKit
functions and utilities to derive the maximum benefit and flexibility from the WinRunner®
test tool.
While this ToolKit was
developed specifically for use with WinRunner®, the concepts employed are platform
independent and can easily be applied to any other test tool having a
"scripting" capability (e.g. Segue's SilkTest, SQA's TeamTest, etc.).
The ToolKit for
WinRunner® allows the tester to accomplish the following:
 |
Quickly
generate Automated Tests without an extensive knowledge of WinRunner's Test Script
Language; |
|
| |
 |
Develop
scripts that are "robust", in that they will not stop running due to an
application error, a WinRunner error, or a test script error (allows "unattended
testing"); |
|
| |
 |
Develop
Automated Test Scripts for more than one application (tests can be developed for any
number of applications of different types without having to "re-invent" the
script development process); |
|
| |
 |
Easily
develop "customized" functions and utilities using the ToolKit as a framework
(if familiar with developing Automated Test Scripts using Mercury Interactive's Test
Script Language); |
|
|
|
 |
Create
customized "Test Reports" for each test and "Summary Reports" for each
scenario or series of tests rather than relying solely on WinRunner's® Test Report;
|
|
| |
 |
Assume
total control of the test tool and essentially make it do whatever is required
to test an application; |
|
ToolKit Functions:
"ToolKit Functions" are basically compiled Automated Test Scripts written in
TSL using the rules and conventions for writing functions. They are written to accomplish
certain basic tasks that are repeated often. These functions can be invoked or executed
from other Automated Tests or from other functions. The ToolKit currently contains the
following types of functions:
 |
"Generic"
functions: can be used for testing any application on any platform. |
|
|
 |
"Windows-Specific
functions: can be used for testing any object-oriented application. |
|
|
 |
"Language-Specific"
functions: used for testing object-oriented applications developed under a specific
language (e.g. SQL [Gupta] Windows, Visual Basic, PowerBuilder, etc); |
|
|
 |
"Terminal
Emulator" functions: used for testing AS/400 or Mainframe applications via terminal
emulation (e.g. Wall Data's RUMBA®). |
|
|
The ToolKit updates WinRunner's "Function Generator" Menu with the Function
descriptions whenever a particular set of Functions is loaded. This allows the
tester to insert functions into scripts using the built-in "Function Generator".
Automated Testing also requires the development of Application-Specific
functions, or "customized" functions developed for the specific Application
Under Test. These are not part of the ToolKit, but are
developed as needed as part of the Automated Testing Process for an application.
ToolKit Utility Scripts:
"Utility" scripts are similar to Functions, in that they are used to perform
repeated tasks. They are written as "scripts", however, and therefore conform to
the rules and conventions that apply to scripts rather than to functions. They are not
compiled. They are "called" by other scripts, and operate like
"subroutines". The ToolKit currently contains the following types of Utility
scripts:
 |
"Generic"
Windows utilities: used for testing any object-oriented application; |
|
|
 |
"Language-Specific"
utilities: used for testing object-oriented applications developed under a specific
language;. |
|
|
 |
"Terminal
Emulator" utilities: used for testing AS\400, ACP/TPF, and other Mainframe
applications via terminal emulation; |
|
|
 |
"Application-Specific"
or "customized" utilities may also need to be developed for the specific
Application Under Test; |
| |
|
A Total Testing Solution
The ToolKit for WinRunner®, combined with an effective,
user-friendly Data-driven Automated Testing Methodology, provides a Total Testing Solution for the WinRunner® product.
Update: 09/01/2001 |
| |
| The "ToolKit for WinRunner" is not a "shrink-wrapped" or out-of-the-box type of
product. Rather it is a combination of automated testing methodology, and WinRunner
scripts and functions written in TSL (Mercury Interactive's Test Script Language) that
will undoubtedly need to be modified to a greater or lesser extent depending on the
idiosyncrasies of the particular application(s) being tested. This is essentially
the collection of scripts, functions, and documentation require to implement the
methodology described in my White Paper: "Totally
Data-Driven Automated Testing".
I use this "ToolKit" in the normal
course of my consulting practice in order to implement spreadsheet-driven or table-driven
automated testing solutions for my clients. Since they are paying for consulting
hours, they are given the "ToolKit" for free.
However, due to the
ever-increasing demand for this material, I am currently in the process of developing the
documentation required for any WinRunner Specialist who is experienced in writing TSL
(Mercury Interactive's Test Script Language)
to implement this
solution themselves, with little or no assistance from me. This will be offered as:
"The ATS Guide to Totally Data-Driven
Automated Testing" |
(or some similar catchy phrase) |
This package will not include any sort of guarantee or claims of any kind, nor will it be
"licensed". It will, however be copyrighted. This package will
include:
Documentation
on the various theories and methodologies relating to Software Testing Automation;
Complete
step-by-step documentation on how to implement a "Totally Data-Driven" Automated
Testing Methodology;
"Sample"
functions and utility scripts written in TSL that you can choose to use as-is or to modify
in any manner you choose;
A tutorial
on using the "sample" scripts to create a "Test-Plan Driven" test;
Complete
documentation of all "sample" scripts and functions will be included;
Optional
"Technical Support" and/or consulting will be available.
I will announce the availability of this package on this page. I had been posting
a series of ever-changing dates in an attempt to predict when this would be
available. This will no longer be done, as it is of no particular use.
It will be ready when it is ready. |
| |
| Updates will be posted here,
so keep checking back! |
| |
| NOTE: I have been
receiving an increasing number of requests for this material in which the sender has
somehow concluded that this material will be offered for "free". This is
not, and has never been the case. At the moment, since the documentation has not
been completed, there are three ways to obtain this material: |
| 1. |
You can have me come out to
your site (USA & Canada only) for a week-long "seminar" during which I will
instruct you or your personnel in how to implement this solution, and help you do it.
The charge for this is $3500 (USD) + expenses. |
|
| |
| 2. |
You can download the material
as-is and have me help you out "long-distance" by telephone and internet.
This usually requires 1-2 hours a day for 1-2 weeks, depending on your level of expertise
with WinRunner. The charge for this is normally $1500 - $2000 depending on how many
hours of assistance you require. |
|
| |
| 3. |
If your company is located
within the greater Los Angeles area and you require a WinRunner consultant, then you can
just as well contract with me as with someone else. I always bring the
"ToolKit" with me and use it as a matter of course regardless of the solution
required for your particular circumstances. View
Resume and/or Request
Rate Information |
|
| |
| When the "ToolKit"
is available for purchase by itself, it will probably be priced around $500. The
price has not been decided, so don't hold me to that figure.
In any case, this will not include any consulting assistance, which may be
purchased at an hourly rate or by choosing either of the above options. |
| |
|
| |
| If you are interested in being notified of the availability of this material, or if you want to discuss any of the consulting options listed above, send me a note
to get on the notification
list. |
| |
|