Mr. Cluey : Library : KBReport

KBReport

Our tale begins with the automation of a tutorial. If you have a good tutorial for your software, I suggest that this be the first thing that you automate. Among other things, it gives you a legitimate first milestone for each release. For no particularly good reason, this tutorial called for coloring something fuchsia. So that's how it was scripted.

Which was fine until, through gremlins yet undiscovered, a bug was introduced on one of our test machines; the color dialog suddenly decided to spell the word fucshia. Well, so much for the script....

Of course, the easy answer was to change the color to red (much easier to spell). A more reasonable solution is to fix the broken file on the test machine. But this is an excellent opportunity to recognize that work arounds are going to be part of your scriptware, and that you should be prepared to code them intelligently. This is, most of you will realize, why I mentioned this whole problem in the first place.

Scripting Workarounds

Stage one is to determine what the workaround is going to be. If it turns out to be a maajor headache to devise the scriptware to work around the problem, you are probably dealing with a test blocker, which is really beyond the scope of this discussion. The tutorial begins with a fairly simple workaround.

do
   ColorDlg.ColorList.Select("Fuchsia") ;
except
   ColorDlg.ColorList.Select("Fucshia") ;

Now, this fragment has two problems. The minor problem is that it conceals the bug. The script is supposed to take the first path. If it doesn't, then something has gone awry, and we want to have that reported. The major problem is that it reports the wrong bug. For example, if the fuchsia entry was missing, the exception handler would report that fucshia was missing. While in this instance the difference is fairly minor, in more complicated exception handling routines, the disparity between the actual problem and the reported problem can be quite large.

Those problems are easy enough to fix. Version two of the code handles this quite nicely.

do
   ColorDlg.ColorList.Select("Fuchsia") ;
except
{
   ConfirmException( E_ITEM_NOT_FOUND ) ; // See Except
   if ListFind( ColorDlg.ColorList.GetContents (), "Fucshia" ) == 0
   {
      reraise ;
   }
   
   LogError( "Bug #1234 Color Fuchsia not found.  Spelling error." ) ;
   ColorDlg.ColorList.Select("Fucshia") ;
   
}

Much better. ConfirmException ensures that only the specific problem we are trying to trap gets trapped. The exception handler first verifies that the work around will succeed - if not the original exception will be raised. The message reported lets us know where the bug tracking number, so later in later runs we will no that the bug is already in the system and may need to be reopened.

There is still a minor problem. This reporting mechanism really isn't satisfactory. We aren't trying to log an error. We are trying to bring to light that this is a problem which has been seen before. At the very least, this calls for a wrapper function.

The wrapper that I use is KBReportKnownBug, defined in kbreport.inc. By using the function that explains what I want, instead of that which merely does what I want, I make the life of the maintenance programmers of the future just a little bit easier.

[Note: The kbreport.inc file is included in the QAPlib which can be downloaded - click here - KZ]

do
   ColorDlg.ColorList.Select("Fuchsia") ;
except
{
   ConfirmException( E_ITEM_NOT_FOUND ) ; // See Except
   if ListFind( ColorDlg.ColorList.GetContents (), "Fucshia" ) == 0
   {
      reraise ;
   }
   
   KBReportKnownBug( "Bug #1234 Color Fuchsia not found.  Spelling error." ) ;
   ColorDlg.ColorList.Select("Fucshia") ;
}

Lovely, and just what the doctor ordered.

Implementing KBReportKnownBug

KBReportKnownBug does not require a particularly sophisticated implementation. This fragment does an adequate job.

KBReportKnownBug( string sMsg)
{
   // A bug is an error, right?
   LogError( sMsg ) ;
}

I wanted a little bit more control over reporting than that. What I had envisioned is one test machine, devoted to reporting new problems, next to a machine reporting every problem that it found, yet running the same collection of scripts.

So to do this, I added a routine to the configuration system ( see config for more on this ) to specify, during startup, which particular reporting mechanism would be used during this run. The downside of this implementation is that it still deals in absolutes - all known bugs are treated equally. The general model does hold for more sophisticated systems, though. Once the framework is set up, changes are pretty simple to make.

KBReportKnownBug (STRING sMsg)
{
	string sTempMsg = "KBR: " + sMsg ;
	
	switch( kbrState )
	{
		case KBR_ERROR:   LogError( sTempMsg ) ;
		case KBR_WARNING: LogWarning( sTempMsg ) ;
		case KBR_MSG:     Print( sTempMsg ) ;
	}
}

Further thoughts

KBReportKnownBug is fairly primitive. A more sophisticated version might be TrackKnownBug, which accepts a bug number, it's severity, perhaps a bug status (Open, Fixed, Deferred), and a short description, and bases its reporting in part on the status of the bug.

Reporting based on status would be pretty slick. Open bugs are reported quietly, because the open status shows that we already know about them, but fixed bugs get reported loudly. If your tracking system makes the distinction anyway, you might differentiate between problems scheduled to be fixed this release and regression bugs.

It would also be a pain in the neck. You would be constantly searching through all of your source code, replacing one status variable with another. You could automate the process, of course, but it gets out of hand quickly. Don't forget that automation is supposed to make your life easier.

Now, if your system supports it, you might be inspired to rewrite TrackKnownBug so that it takes the bug number, invokes your bug tracking system and uses the current status of the bug within the system to determine how to report it. Very nice.

For extra credit, write a Bug reporting function, so that when QA Partner finds problems, it can put them into the tracking system for you. My, my; we are getting lazy....