Parsing Resource Files to Create and Maintain an
Automation Test Library
Amy Groves
New England Segue User Group
Each build of a software application under test may introduce, delete, or
modify GUI-object tags referenced by automated test scripts. Failing test
scripts must then be debugged so that error reports caused by outdated tags
can be isolated from bugs in the application under test. Debugging time can
be reduced via a utility that parses each build?s resource files into
header files referenced by the test scripts.
I. SCOPE AND AUDIENCE
This paper is intended to promote better use of test automation by
describing a problem common to automation engineers and suggesting a
solution to that problem. The information presented is general enough to be
applied to any software development company. So as not to reveal
proprietary information, the paper presents no company-specific data. The
intended audience is composed of engineers who are familiar with QA Partner
testing software and the 4Test scripting language. Knowledge of the
following is helpful but not required: Windows 32-bit programming, the C
programming language, and the Microsoft Visual C++ compiler. It is assumed
that the application under test has been developed using Visual C++ 4.0 or
above, although testers of Visual Basic applications should be able to
follow the general procedures described in this paper as well.
II. TERMINOLOGY
Windows resources are parts of an application with which users
interact. Examples include accelerator (shortcut) keys; bitmaps for
graphics, cursors, and icons; dialog boxes; menus; string tables; toolbars;
resources containing version information; and custom resources. For each
resource, a Windows compiler such as Visual C++ will generate an
ASCII-formatted resource definition composed of the following: a resource
script or resource file (.rc) that contains the definition of the resource,
and entries into a C-style header file which assigns numeric values to
constants referenced by the resource file.
4Test include files will be referred to as ?.inc? files so as to avoid
confusion with C-style include files. Examples have been written in what is
commonly referred to as the ?4Test Classic,? as opposed to Visual 4Test,
syntax.
III. STATEMENT OF THE PROBLEM
As an application under test grows in size and complexity, so does its
test automation harness. The more extensive the automation harness, the
longer it takes to analyze test failures and differentiate failures due to
bugs in the application from failures due to errors in the harness. This
analysis causes a considerable delay in reporting bugs. In an attempt to
accelerate bug reporting, QA managers may find themselves allocating
increased person hours to maintenance of the harness. Overall, a
degradation in the cost-vs.-payback ratio of automated testing is evident.
Reducing the incidence of test failure due to errors in the harness is one
strategy for correcting this downward spiral. Since a high percentage of
these failures is due to out-of-date 4Test tags, a utility which updates
these tags automatically for each new build will be extremely valuable.
Such a utility is described in the next section.
IV. DESCRIPTION OF THE PARSER
At the highest level, the parsing utility is based on simple looping.
Various algorithms will work; one follows:
Until no more resource files
If corresponding .inc file doesn?t exist
Create .inc file with blank parsable area
Else
Erase parsable area of .inc file
Until end of resource file
Read string from resource file
If string matches a predefined grammar rule
Translate string
Output new string to parsable area of .inc file
In order to understand the structure of the parser input, it is
necessary to understand how resource files are generated. Typically, the
application developer draws the resource within the Visual C++ Developer
Studio, which includes several resource editors, one of which will generate
a file similar to the one below, which we will call mydialog.rc:
//
---------------------------------------------------------------------------
IDD_MYDIALOG DIALOG DISCARDABLE 0, 0, 186, 95
STYLE DS_MODALFRAME | WS_POPUP | WS_CAPTION | WS_SYSMENU
CAPTION "My dialog caption"
FONT 8, "MS Sans Serif"
BEGIN
DEFPUSHBUTTON "OK",IDOK,27,74,50,14
PUSHBUTTON "Cancel",IDCANCEL,97,74,50,14
LTEXT "Here is my static text.",IDC_STATIC,81,38,68,8
CONTROL
"MyCustomBitmap",IDC_CUSTOM_BMP,"",WS_TABSTOP,27,25,41,32
END
//
---------------------------------------------------------------------------
Based on the above input, the paser produces the following output:
//
---------------------------------------------------------------------------
// filename: mydialog.inc
// PARSER BEGIN
window DialogBox Mydialog
{
tag tgMydialog;
PushButton Idok {tag "${tgIDOK}";}
PushButton Idcancel {tag "${tgIDCANCEL}";}
StaticText Idc_static {tag "${tgIDC_STATIC}";}
CustomBitmap Idc_custom_bmp {tag "${tgIDC_CUSTOM_BMP}";}
// PARSER END
}
//
---------------------------------------------------------------------------
If the eight-character resource filename is unique within the source
code stream, the same filename can be used for the .inc file. The dialog
name IDD_MYDIALOG is output to the .inc file as Mydialog. Then the parser
must transform the resource file statement:
CAPTION ?My Dialog Caption?
into the .inc file statement:
tag ?My Dialog Caption?;
or better still, to the following statement:
tag tgMydialog;
appending one line:
STRING tgMydialog = ?My Dialog Caption?;
to a separate .inc file that stores all string assignments, thus
facilitating internationalization of the harness. The four control
declarations are parsed next, with Win32 classes mapped to 4Test classes as
shown. The final control, Idc_custom_bmp, illustrates the necessity for the
parser to look beyond the general CONTROL class specification and map the
IDC_CUSTOM_BMP style attribute to the 4Test user-defined CustomBitmap
class. Because resource control declarations also contain styles such as
IDOK, to which numerical equivalents are assigned in a header file, the
parser must read in that header file line by line, translating:
#define IDOK 100
into the following format:
const IDOK = 100;
and appending that output to a separate .inc file.
The first time the parser creates an .inc file containing a window
declaration, it inserts the PARSER BEGIN and PARSER END statements so that
only the lines falling between these two markers will be updated by further
parsing. In this way, a parent statement (notably absent from the resource
file) and methods can be keystroked after the close of the parsable area
and preserved for future use.
The parser can be built as a separate executable or as several
separate executables, and run via batch file or makefile. It is thus
possible to incorporate .inc file updates as part of the application?s
build process. If the automation harness is under version control, a batch
file may be written to lock, update, check in, and unlock the parsed files.
Developers implementing this parser on Windows NT are urged to
consider using either the C language or a combination of C, lex, and yacc.
Originally written as Unix tools, lex and yacc have been ported to NT by
Mortice Kern Systems as part of the MKS toolkit. Lex divides input into
meaningful units, or tokens, and yacc reports each instance in which a
token matches one of the parser?s grammatical rules. Although most
developers will need to learn to use the MKS tools, they will be rewarded
with a smaller parser in terms of overall lines of code. Execution time is
anticipated to be short enough as to be a trivial consideration, regardless
of the choice of development tools.
V. OBSERVATIONS
The coding conventions used by developers may create a number of
problems for automation engineers. Generally, one or more of the following
three solutions may be used to address these problems: implementation of an
exception handler in the parser, close work with developers, and the
introduction of 4Test workarounds.
One problem occurs when the caption statement in the resource file is
overridden at runtime, as in the following examples:
CAPTION "My <insert> Dialog"
CAPTION "My Dialog Box" // overwritten to "My Dialog" at
runtime
CAPTION ""
In the first case, the parser should replace <insert> with a wildcard
character (*). The exception handler should address the second case,
replacing ?My Dialog? with ?My Dialog*?. The exception handler should also
replace the null caption with a single wildcard character. The incidence of
two dialog declarations sharing the same string caption tag presents a
problem only if the tester is using a recording dialog to discover the
variable name assigned to a particular dialog or control. In this case, the
two dialog declarations can be merged into a single .inc-file declaration
including a superset of controls from both dialogs.
A more serious problem is encountered when entire dialogs are drawn at
runtime, so that they aren?t declared in any resource file. The automation
engineer has no choice but to grep the source code for calls to functions
which might draw these dialogs, and keystroke the .inc files declaring
these dialogs. Developers should be made to understand that runtime dialog
generation tends to break automation, and that this drawback should be
considered during code design.
Since more than one dialog may be declared in a resource file, the
parser must map each resource declaration to the correct .inc file
declaration. If a developer adds a dialog declaration to the middle of a
resource file that has already been parsed, the parser will need to
preserve the correct mapping so that manual code additions to the .inc
file(s) will not be lost.
A high incidence of preprocessor directives in the source code can
place serious demands on the parser?s algorithm. The logic must address all
possible combinations of directives. Encouraging developers to minimize
nested #IFDEF statements will save much parser development time.
For your efforts in addressing the above problems, you may receive
certain unexpected rewards in addition to reducing the amount of time you
and your colleagues spend debugging the automation harness. The parser's
logic may find source code syntax errors that might have caused costly bugs
later. You will find that syntax errors in the parser?s output are easy to
detect via compilation so that they can be fixed right away, before the
updated automation harness is released to other users. In the long run, you
may find that authoring a parsing utility creates opportunities for
professional advancement.
VI. CONCLUSIONS
The lessons learned from this study are hard-won but significant. As
in any development effort, good engineering practices, including code
reviews, group design discussions, and adequate research, are imperative.
Since theoretically any statically-declared object can be parsed, parser
code should be portable and general-purpose in nature so that the scope of
the project can be broadened later. Writing and maintaining a parser is an
expensive project that should not be undertaken without complete support
from development and upper management. Source code access, even if read
only, is crucial. Developers should be alerted to coding practices that
will break test automation, and they should be reminded of the link between
design for testability and speedy bug reporting. Automation engineers who
learn and exploit existing development build processes and tools will fare
better than those who attempt to work in isolation.
Numerous possibilities exist for further study and development. Data
might be assembled comparing the time invested in writing and maintaining
the parser to the decrease in time required to detect and report bugs. The
possibility of increasing the parser?s functionality might be investigated;
in particular, the parser could flag occurrences of new, modified, and
obsolete dialogs and controls. Internationalization engineers might modify
existing parsing code slightly and implement a working version of their own
with a relatively small investment. The benefits of the parser are
abundant, as is the work involved in its development and use.
BIBLIOGRAPHY
Anonymous. ?Segue Software, Inc.? 1997. http://www.segue.com/ (4 Nov.
1997).
Anonymous. ?Introduction to C Programming.? 1997.
http://www.iftech.com/oltc/c/c0.stm (4 Nov. 1997).
Chan, Patrick. ?Mortice Kern Systems (MKS) Inc. ? Award-Winning Products
and Solutions.? 1997. http://www.mks.com/ (4 Nov. 1997).
Levine, John R., Tony Mason, and Doug Brown. Lex & Yacc. Cambridge, MA:
O?Reilly & Associates, 1992.
Perry, Paul, et al. Special Edition Using Visual C++ 4. Indianapolis, IN:
Que Corporation, 1996.