Mr. Cluey : Palindromic Frame Model : First Essay

Palindromic Frame Model

This was originally posted on comp.software.testing

In article <6p8qr0$8rk$1@nnrp1.dejanews.com>, faren451@my-dejanews.com says...
> I'm using silk test 2.1 to test a web based product where the interface and
> functionality will be gradually over time.  I'm fairly new to using this tool
> and was wondering if anyone that has more experience with it might have some
> advice as to what would be a good way to organize testcases in order to
> simplify the maintenance of the scripts.
> 
> i.e would it be better to test the gui separately from cgi?  Test the part of
> the cgi that interfaces with the gui separately from the part which connects
> to a database.
> 
> I don't want to have to go through the code changing a bunch of things just
> because a new field was added to a form for example.

You are absolutely right.  You sure don't.

I have found that the best way to minimize the amount of maintenance caused 
by changes to the application is to isolate the functionality that I am trying 
to test from the interface to that functionality.

Silk already does this to some degree - the usual plan of attack is that you 
record a window declaration for the application that you are going to test, 
then your test script interfaces with the declaration.  Thus you end up with 
a picture that looks something like this

script -> frame | UI -> App

Or, as I will sometimes draw it to help clarify the point
 A' -> B' | B -> A

I've come to refer to this design as the palindromic frame model, which is 
just a fancy way of saying that the layout of our test code should be a mirrored 
reflection of the application that we are testing.

I leave it to the giants of the field to argue whether the following is true, 
but I myself have always felt that your tests should be coded against the 
specification, rather than against the application.  The picture that I have 
drawn above, therefore, isn't quite right.  It should really look like

script ->AppFrame -> UIFrame | UI -> App -> Spec

The pattern is still the same, of course, we've just added an extra layer.

These individual layers can be broken down still further, of course.  
My usual example is opening a file in Notepad.  The volatile way to 
test this is to have your script click on the menu, and type everything 
into the dialog that pops up.  In the palindromic model, 
your script would call NotepadApp.Open(), which would call 
NotepadUI.Open(), which in turn would call NotepadOpenDlg.Open () 
(or possibly SysOpenDlg.Open(), depending upon how much you like 
to re-use code), which would do all of the work.  Now, when somebody 
comes along and changes the way the OpenDlg works (as happened when 
Win32 was sprung upon the world), you only need to change your code 
in one place to get everything up and happy again.

There isn't, to my knowledge, a thing you can do about automating 
the testing of a volatile section; if they keep changing the UI on 
you, you are going to have to keep changing the way that you test it.  
But if you isolate things carefully, a UI change won't force a change 
in your *other* tests, because those will be insulated by the UI layer.

There are some other advantages as well - for instance, if you write 
your scripts to be testing at the spec level, you can begin writing 
those tests as soon as you have a spec, without having to worry yet 
about what the UI will actually look like - you can even drop in a "Stub" 
UI to be used to verify that your scripts area working as you expect them 
to before you ever get to see the "real" application.  

Furthermore, because you are really testing against the functionality, 
rather than against the UI, you are pretty well impervious to even nasty 
UI changes - such as those caused by internationalization.  You can simply 
snap in the new declarations (If you are really clever, you can do it 
on the fly) and away you go.

Another bonus is that your tests will look like manual test scripts.  
They can be handed to manual testers while you are busy getting the under 
layers written.  Heck, you may even be able to get your non-technical 
people creating automated tests for you, because for the most part the 
test layer will be written in the language of the application, as it were.

So to return to your questions, yes you should be testing the UI 
seperately from the cgi; and the cgi to backend interface separately 
from the cgi to front end interface.  Compartmentalize everything that 
you can, so that changes to the back end don't kill your tests of the 
front end, and vice versa.  Time and other pressures will force you 
to make some comprimises, I suspect, but I strongly believe that this 
is the design you should be shooting for.