Previous Next Contents Generated Index Doc Set Home


CHAPTER 14

Sun WorkShop Visual Replay





Introduction

Sun WorkShop Visual Replay can record and playback any Xt based application.

In record mode, Sun WorkShop Visual Replay creates a script containing a high level description of the user's actions e.g. "push hello_button, type Hello World".

In playback mode, you can check the state of any widget in the application and control the rate of playback. The actions in the script are replayed exactly as if the user were sitting at the keyboard.

Sun WorkShop Visual Replay has a user extensible command set which is powerful, easy-to-use and very flexible. It can be deployed in many ways:

Record and playback Motif applications
Produce demonstrators of your product
Debug your application
Recreate problems found in an application
Develop tutorials for your product
Automate the testing of an application
No recompilation or relink is necessary and no special test environment is required.

Sun WorkShop Visual Replay is available from the Sun WorkShop Visual "Tools" menu. You can also use the tool from the command line as described in "Recording and Replaying from the Command Line" on page 449.

This chapter starts by providing you with a description of the use of Sun WorkShop Visual Replay together with some simple tutorial examples to help you become acquainted with its use. "Extending the Sun WorkShop Visual Replay Widget Set" on page 460 and "Adding your own Sun WorkShop Visual Replay commands" on page 473 describe how to extend the capabilities of Sun WorkShop Visual Replay.

Appendix  A, "Sun WorkShop Visual Replay Command Syntax", starting on page 815 provides detailed descriptions of the syntax of Sun WorkShop Visual Replay scripts.


Recording and Replaying Java Applications

Sun WorkShop Visual Replay can be used for Java applications. To do this, you should specify the Java interpreter as the first application and your target application afterwards, as explained in "Debugging with Sun WorkShop Visual Replay" on page 460. That section describes the use of indirections when Sun WorkShop Visual Replay is used from the command line. Thus, to record or replay a Java application, you would have to specify the Java interpreter too:

visu_replay java MyJavaProgram

This applies to Java applications but not to applets. For more information on Java code generation, see Chapter 10 "Designing For Java".


Before Using Sun WorkShop Visual Replay

For successful operation of Sun WorkShop Visual Replay, the Motif application you wish to record must have been dynamically linked with the Xt library (libXt). On many UNIX implementations, you can find out whether the application has been dynamically or statically linked with libXt by typing:

ldd AnApplication

If the output mentions libXt, the application has been dynamically linked with the Xt library and can be used with Sun WorkShop Visual Replay. If this library is not present, the application has probably been statically linked with the Xt library. You will have to relink your application with the Xt shared library if you want to use Sun WorkShop Visual Replay.


How to Invoke an Application with Sun WorkShop Visual Replay


Note - If you are keen to get started straight away with Sun WorkShop Visual Replay, you may wish to skip this section and move directly to "Tutorial" on page 445.
When you select "Sun WorkShop Visual Replay" from the "Tools" menu, a dialog is displayed requesting the name of the application you wish to record or replay. This dialog is shown in Figure 14-1.

FIGURE  14-1 Capture/Replay Application Prompt

Enter the name of the application in the text box labelled "Executable". If you are unsure of the application's name, or where to find it on your system, press the button labelled "Executable". This produces a file selection box containing an extra scrolled list, as shown in Figure 14-2.

FIGURE  14-2 Capture/Replay File Selection Box

Each item in the extra scrolled list labeled "Path" is a directory from your PATH. Selecting an item from this list displays the contents of that directory in the "Files" list.


Note - The extra scrolled list uses the PATH set up for you when you ran Sun WorkShop Visual and may contain some extra directories required by Sun WorkShop Visual. When you exit Sun WorkShop Visual your PATH is the same as it was before running the application.
When you select an entry from the "Files" list and "OK" the dialog, the entry is placed in the "Executable" field of the Capture/Replay Application Prompt dialog. Enter any flags or arguments for the application in the text box labelled "Arguments" in this dialog. When you press "OK", the application is run with Sun WorkShop Visual Replay.

Two points need to be made here:

  1. The Sun WorkShop Visual Replay dialog is part of the application and will be dismissed when you exit from the application.
  2. Sun WorkShop Visual Replay only monitors what happens in the application - it does not and cannot affect the operation of the application other than by simulating operations in playback mode.

What Gets Recorded

Sun WorkShop Visual Replay has been designed as an efficient way of exercising Motif interfaces with the emphasis on portability and clarity of description.

Sun WorkShop Visual Replay focuses on recording navigation between widgets within an application and the user interaction with those widgets. The following information can be recorded and replayed:

actions on all widgets
displaying/closing sub-dialogs.
use of keyboard accelerators, modifiers.
all keyboard input (text, arrow keys, delete etc.).
all mouse button actions - including which mouse button has been pressed.
Sun WorkShop Visual Replay has not been designed as a general-purpose X testing engine and, consequently, there are some aspects of the use of an application which Sun WorkShop Visual Replay does not record. However, provision is given for you to extend the capabilities of Sun WorkShop Visual Replay. This is discussed in "Extending the Sun WorkShop Visual Replay Widget Set" on page 460 and "Adding your own Sun WorkShop Visual Replay commands" on page 473.


The Sun WorkShop Visual Replay Interface

The Sun WorkShop Visual Replay dialog appears to the side of your application. A copyright message is also shown on standard error when Sun WorkShop Visual Replay starts up.

If the copyright message does not appear, your application has probably not been dynamically linked with the Xt library (see "Before Using Sun WorkShop Visual Replay" on page 436).

The Sun WorkShop Visual Replay dialog is shown in Figure 14-3.

FIGURE  14-3 Sun WorkShop Visual Replay dialog

You can also display this dialog when running Sun WorkShop Visual Replay from the command line. This is explained in "Recording and Replaying from the Command Line" on page 449.

This dialog has two pages - one for Sun WorkShop Visual Replay (Replay) and one for Sun WorkShop Visual Capture (Capture). You can change between pages by selecting from the option menu labelled "Page". For details on the dialog when you are using Sun WorkShop Visual Capture see "Sun WorkShop Visual Capture" on page 428.


Functions and Operations

Once the Sun WorkShop Visual Replay dialog is displayed, you can begin to record/replay scripts straight away. All record/replay actions take place using the Sun WorkShop Visual Replay button panel which is shown in Figure 14-4.

FIGURE  14-4 Sun WorkShop Visual Replay button panel

The buttons are described below:

Record

records user actions in the application from the current position in the selected script. If record is pressed after stopping a script, it will overwrite the script from that point on. Pressing record at the end of a script will append to it.

Insert

records user actions in the application at the point where the script was stopped. Subsequent actions in the script are preserved.

Rewind

rewinds the selected script to the beginning. To replay the script exactly, you may have to reset the application to the state from which the recording was started.

Stop

stops the playback of a script.

Play

plays the selected script from the current position in the script until either the script is stopped or reaches its end.

Single step

plays the next command in a script.

Pause

pauses a record or playback. Press the button again to continue.

Only valid buttons can be selected; all other buttons are grayed out.

Before you have created any scripts, the only button you can press is "Record". This creates an "unnamed" script. Once you have created a script, you can "Rewind", "Play" and "Single step" it.

The "Insert" button becomes active when the script is stopped or paused. The insertion process is described more fully in "Inserting in a Script" on page 449.


Creating and Naming Scripts

Press the "New Script" button to create an empty script. To name or rename a script, do the following:

  1. Click on the associated icon in the Sun WorkShop Visual Replay dialog
  2. Enter the name in the New Script text field and press Return
If you enter the same name as that of an existing script, a number is appended to the newly named script to differentiate between it and the original.


Note - If you have no scripts in the Sun WorkShop Visual Replay dialog, pressing "Record" will create a new "unnamed" script automatically.

Selection and Status Indicators

The currently selected script is highlighted in the Sun WorkShop Visual Replay dialog.

The Sun WorkShop Visual Replay status indicator shows you whether you are recording or replaying and where in the script you are. If the status indicator is red, it indicates that you are recording. Otherwise you are replaying. Figure 14-5 shows the possible states of the indicator:

FIGURE  14-5 Sun WorkShop Visual Replay Indicator States

The last button you selected has a red line above it in the button panel.


Monitoring

The "Monitor" button displays a log of the actions you are taking while recording and replaying. Comments indicating the start and end of a record or replay session are inserted automatically by Sun WorkShop Visual Replay as demonstrated in Figure 14-6.

FIGURE  14-6 Sun WorkShop Visual Replay Monitor Window


Inserting Extra Commands

As well as actions, you can also add non-application commands and comments to a script. This can be done by editing the script by hand or via the Sun WorkShop Visual Replay interface. This section describes how to edit the script from the interface.

First stop the script at the point where the additional commands are to be placed. To place extra commands at the start of the script, you must first rewind it. To place commands at another point in the script, single-step to that point.

Next press the button labelled "Extra Commands". This displays a text edit window into which the extra commands or comments can be entered. This dialog is shown in Figure 14-7.

FIGURE  14-7 Extra Commands dialog

If the "Enter as comment" toggle is set, the contents of the dialog are treated as comments. Each line is prepended with a `#' character in the script.

The "Run" button executes the commands in the dialog independently of the recorded script. Use the Monitor window to see the commands being executed. Once you are satisfied with the commands, press the "Add" button to store them in the script.

Press "Clear" before entering additional commands or comments. This removes the information from the Extra Commands dialog - it has no effect on the contents of the script.


Changing Replay Speed

The fast/slow slider on the Sun WorkShop Visual Replay dialog allows you to change the speed at which the selected script is replayed. By default, the script is played at the maximum speed.


Application Modal Dialogs

If your application runs an Application Modal dialog, you will not have access to the Sun WorkShop Visual Replay interface until you have closed the dialog. This means that you cannot stop recording or replaying within the dialog. In single-step mode, all actions within an Application Modal dialog are treated as a single step.


Saving and Accessing Scripts

By default, the scripts you create in the Sun WorkShop Visual Replay dialog are stored in a temporary unnamed directory.


Note - Unless the environment variable XDS_KEEPDIR is defined, the temporary directory (and its contents) is removed automatically when you exit the application. If XDS_KEEPDIR is set, the temporary directory and its contents are stored in /tmp/XDS_SAVE. If you want to preserve your work, you should be working in a named directory (see below).
Use the "Save As" option from the Sun WorkShop Visual Replay Directory menu to save the current directory under a new name.

Use the "Open" option from the Directory menu to access scripts from another directory. The "Save As" option can also be used to rename the currently opened directory.

Using the operations in the "Edit" menu, scripts can be cut or copied from one directory and pasted into another. The "Clear" command deletes the selected script.

By convention, record scripts are given the filename suffix ".xds" in the file system. Note however that this suffix is not used to label the scripts in the Sun WorkShop Visual Replay dialog.


Tutorial

This section is a set of step-by-step instructions which demonstrates how to use Sun WorkShop Visual Replay to record interaction with the visu_config tool and then replay those actions.


Note - The tutorial requires no knowledge of visu_config. If, however, you would like more information on this tool, refer to "visu_config - the Main Dialog" on page 639.
  1. Select "Sun WorkShop Visual Replay" from the "Tools" menu.
  2. Type: visu_config into the Executable field of the Capture/Replay dialog and press "Ok".
  This runs visu_config and displays the Sun WorkShop Visual Replay dialog alongside it.
  3. Press the "New Script" button.
  This creates an "unnamed" script.
  4. Enter a name for the script in the New Script text field, followed by a carriage return.
  The name of the script is changed accordingly.
  5. Press the "Monitor" button.
  This brings up a dialog showing a log of all the actions for the session.
  6. Press the Record button, as shown in Figure 14-8.

FIGURE  14-8 Record Button

  7. In visu_config, perform the following actions:
  a. Enter: one in the Selection text field and press Return.
  The name is added to the "Families" list.
  b. Double click over the name "one" in the Selection field, type: two and press Return.
  The Families list now contains two entries.
  c. Click on one in the Families list and press the "Edit" button.
  The "Widget Classes" dialog is displayed.
  d. Enter: WidgetOne in the Selection text field and press Return.
  The name is added to the "Widget classes" list.
  e. Double-click "WidgetOne" in the Widget classes list.
  This displays the Widget dialog.
  f. Press the "Close" button in the Widget dialog.
  g. Press the "Close" button in the Widget Classes dialog.
  h. Select the "Stop list" option from the "Edit" menu in the Families dialog.
  This displays the Stop list dialog.
  i. Press the toggles labelled "Pulldown Menu" and "Text Field" in the Stop list dialog.
  j. Press the "Apply" button followed by the "Close" button.
  k. Select "New" from the File menu in the Families dialog.
  The "Save changes" warning dialog is displayed.
  l. Press the "No" button in the Save changes dialog.
  8. Press the "Stop" button
  The "Record" and "Rewind" buttons become sensitive. All the other buttons become insensitive.
A file has been created containing a record of your actions. This file can be replayed at any time. For the purposes of this tutorial, we are going to play it back straight away.

  9. Press the "Rewind" button.
  The record, insert, play and single step buttons become sensitive.
  10. Press the "Play" button.
  You can now see what you have recorded. Using the fast/slow slider in the Sun WorkShop Visual Replay dialog, you can change the rate at which your session plays back.
  11. Press the "Rewind" button.
  12. Press the "Single step" button.
  Using this button you can single step through each command in the record script. This is more informative if you have the Monitor window on the screen. As each step is replayed it is printed in the Monitor window.
  13. Exit visu_config.
  Select "No" when you are asked if you wish to save the changes. The record session ends when the application exits. The Sun WorkShop Visual Replay dialog is also dismissed. This is because the dialog is, in effect, part of the visu_config program.

Note - Unless the environment variable XDS_KEEPDIR is defined, the temporary directory (and its contents) is removed automatically when you exit the application. If XDS_KEEPDIR is set, the temporary directory and its contents are stored in /tmp/XDS_SAVE. If you want to preserve your work, you should be working in a named directory (see below).

The Contents of the Script

The example above produces the following script, annotated to show the steps in the tutorial. This script introduces some 90% of the Sun WorkShop Visual Replay syntax.


Note - The file you created may not be exactly the same as this one because you may have performed the actions in a slightly different order or you may have made mistakes and gone back to correct them. All of this is recorded.

Inserting in a Script

You can insert at the beginning of a script or partway through it (i.e. during a single step sequence).


Note - You can find out exactly where you are in the script if you have the "Monitor" window open.
To add to a script:

If you are at the beginning of a script, press the "Insert" button.
If you partway through a script, press the "Stop" button and then the "Insert" button.
In both cases, then continue using the application.

Pressing "Insert" is the same as pressing "Record" except that whatever you do in the application is inserted into the existing script at the current point. When not in Insert mode, pressing "Record" will overwrite whatever was in the script.


Note - Remember when inserting actions into a script that script must be able to continue after the insertion. If this cannot be done, the replay will stop at that point.


Recording and Replaying from the Command Line

Sun WorkShop Visual Replay is supplied as a stand-alone application which can be run from the command line both for recording and replaying scripts.


Using Sun WorkShop Visual Replay to Record Scripts

Sun WorkShop Visual Replay (when used to record user actions) is supplied as a stand-alone application called visu_record.

Type: visu_record -x to display basic information about the tool.

The following line shows how to use visu_record:

visu_record -f MyRecordScript AnApplication

MyRecordScript is the name of a file into which a script recording the session will be saved. You do not have to supply this parameter. If you do not, the script is written to standard output. AnApplication is the name of the application you wish to record. The -i flag tells Sun WorkShop Visual Replay that you wish to use the tool interactively. In this case, the Sun WorkShop Visual Replay dialog is displayed as described in "The Sun WorkShop Visual Replay Interface" on page 439.


Using Sun WorkShop Visual Replay to Play Back Scripts

Sun WorkShop Visual Replay, when used to play back recorded scripts, is supplied as a stand-alone application called visu_replay.

Type: visu_replay -x to display basic information about the tool.

The following line shows how to use visu_replay:

visu_replay -f MyRecordScript AnApplication

MyRecordScript is the name of a file containing the script of the recorded session. You do not have to supply this parameter. If you do not, the script is read from standard input. AnApplication is the name of the application you wish to rerun. The -i flag informs Sun WorkShop Visual Replay that you wish to use the tool interactively via the Sun WorkShop Visual Replay dialog.


Getting the Most from Sun WorkShop Visual Replay

This section describes the uses to which Sun WorkShop Visual Replay can be put and discusses:

preparing rolling demonstrations
taking screen dumps
testing
debugging
This list is neither definitive nor exhaustive - it serves only to demonstrate the wide-ranging capabilities of Sun WorkShop Visual Replay.


Preparing Rolling Demonstrations

A script prepared using Sun WorkShop Visual Replay can be run in a "continuous loop" using a simple shell script, as shown below:

while (true)

	{
		visu_replay -f mydemo.xds myapplication
	}
	end

Note - When preparing such a rolling demonstration, always ensure that the last part of your script has commands which place your application in a state from which it can be re-run.


Taking Screen Dumps

Taking screen dumps of an application can be a tortuous process - particularly if the application is constantly subject to change. Sun WorkShop Visual Replay allows you to create screen dumping scripts which can be reused at any time. And because the screen dumping process is now automatic, the cost of producing them falls dramatically.

A screen dumping script consists of a set of actions to prepare the application for the screen shot followed by non-application commands which actually do the screen shot. In the example script fragment shown below, a screen dump of the current_shell dialog is taken:

in current_shell

		setenv ID WindowFrame(current_shell)
		shell xwd -id $ID -out /tmp/current_shell.xwd
The last two lines are extra, non-application, commands. The first sets the variable ID to the current shell window, including its window decorations. The second uses the xwd command to get a snapshot of the shell window and store it. Of course, you can substitute xwd with any other screen dumping command of your choice. The keywords used for setting variables are discussed in "Non-application operations" on page 829.


Testing

Sun WorkShop Visual Replay is a simple-to-use, portable, and powerful widget-based testing tool. It is intended to provide a testing solution across the whole range of platforms that are supported by Sun WorkShop Visual.


The Role of Widget-based Testing

Most Motif/Xt programming involves reusing the Motif widgets, and using the X Toolkit. Sun WorkShop Visual Replay testing focuses on the Xt widget hierarchy, both for controlling a test sequence and for checking whether a test has succeeded.

It is important to note that you are not checking whether the widgets themselves are correct - only that user interaction with those widgets produces the desired results within your application.


Note - This testing technique and strategy is highly resistent to test "rot". Your test results should be the same, whatever the size, shape or quality of the display being used. Tests will only need to be added or updated if the application itself changes. And of course these tests will soon detect any changes which have not been reported to the tester!
Not all testing can be automated in this way. There will always be a need to visually inspect an application to check whether it looks right or whether any graphics programming (e.g. in drawing areas) has worked. While there will always be a requirement for looking and thinking, the widget-based testing strategy ensures that you can focus your attention on those few parts of the application that need it.


The Approach to Testing

Experience has taught us that there are three graduated approaches to the production of a testing script:

recording and replaying pre-recorded scripts
splitting large tests into smaller fragments
data-driven testing

Recording and Replaying Pre-recorded Scripts

This is the simplest way of checking that user actions can be replayed exactly as they were recorded. However, the scripts can become very large and troublesome to maintain. It can also be difficult to work out which part of a test is failing. More importantly, any change to the application will mean that the whole script will need to be re-recorded.


Script Fragmentation

Here a large script is split into small, self-contained scripts each of which exercises an identifiable part of the application. Since this is such an effective testing technique, we have provided a detailed example in "Using Testing Macros" on page 457. Each fragment is expanded using a preprocessor (e.g m4 or cpp), or any programming language you feel comfortable with. This allows you to build scripts such as:

StartApplication()

OpenFile(foo.c)

CloseApplication()

Your preprocessor, interpreter or compiler would then translate these fragments into a full Sun WorkShop Visual Replay command sequence.

This simple strategy takes you away from "step-by-step" programming, and your test scripts will be far more manageable.

The language you use for expressing your tests should be carefully selected. The main criteria should be:

Ease of expression. If it is hard to describe your interface using the language, then your model will be harder to write, harder to read and harder to understand.
Familiarity. If you use a language you are at ease with, then you can concentrate on your model.
Class based languages such as Java or Python are ideal for this purpose. Modelling languages, tailored for symbolic processing, such as Lisp or Prolog are other obvious candidates. Preprocessors such as m4 or even the C preprocessor will get you going very quickly.

Alternatively you may prefer to build your model in the language used by your application. In this way you guarantee that it is always available when you port your software. The only rule of thumb is that if you feel you're writing a program rather than designing a set of tests, there is almost certainly an easier way.

This testing method is appropriate for most small to medium-sized applications. However, for very large applications (and Sun WorkShop Visual is a good example) fragmentation also has its limitations:

You can easily end up with a large number of fragments
Many fragments will be doing similar things in different dialogs - you might have an open fragment for each dialog in an application
The fragments are procedural and prescriptive. While a fragment may contain lots of useful information about a dialog, you can only use it in the way it was intended.
The next sub-section describes how to overcome these problems.


Data-driven Testing

Our experience in devising tests for Sun WorkShop Visual has shown that the most cost-effective way of writing tests is to provide a description of each dialog and then use that description in the tests. Consider the following example where Sun WorkShop Visual's Color Dialog is described:

		ColorDialog.shell			= my_color_shell
		ColorDialog.helpbutton			= color_help
		ColorDialog.applybutton 	= color_apply
		ColorDialog.quit        	= color_quit
The names on the left provide an indirect way of referring to the widgets in a dialog. The names on the right are the specific widget names. Such descriptions could then be used in general purpose routines by simply passing in the name of the dialog, for example:

		CheckHelpFor(ColorDialog)
		Close(ColorDialog)
The definitions of CheckHelpFor and Close are shown below:

#define CheckHelpFor(dialog)

		in dialog.shell
			push dialog.helpbutton
	#enddef	
#define Close(dialog)

		in dialog.shell
			push dialog.quit
	#enddef	
These routines could be used for any dialog with a description such as that listed for ColorDialog, to check that help and close buttons have been provided.

This technique allows you to separate out the description of the interface from the actions which exercise it. It also means that any change to the interface requires only a change to the associated data description - test scripts remain unchanged. If a new dialog is introduced to the application, you simply have to write its description and any non-standard operations which may be performed on or in it.

The biggest advantage of such a strategy is that the description is simple, clear and so close to the design itself that keeping tests in sync with product development becomes a well defined and straightforward exercise.


Checking Test Success/Failure

A good test is one which has been designed to break that part of the application it is checking. The test is successful if the application does not fall over, otherwise it is a failure.

Automated replay, by itself, is a minimal form of testing. If the sequence replays without error, then you have some measure that what was expected did actually happen. It is minimal because it only tests one potential result of a user action.

Consider the action of opening a file. In a minimal test, the expected result would be that the file is opened and everything progresses smoothly. However, this test is by no means complete. You need to consider other (potential) results, e.g.

what happens if the file is not accessible
should status indicators change when the file is loaded
if the file is read-only, is a warning dialog displayed

Using Control Flow and Expressions in a Test Script

The simplest test is one which records a series of actions within your application and then replays the script to duplicate those actions. While successful execution of such a script can give some confidence in your application, you can gain even greater confidence by taking advantage of the extra commands for control flow and expressions provided by Sun WorkShop Visual Replay to enrich a basic script. These allow you to cater for different display types, check widget resource settings, print messages, and much more.

Consider the situation where your application displays a message when it is running on a monochrome display but displays no message when it is running on a full color display.

Clearly, you don't want to have a separate test for each display. Instead, you can insert commands at the point where you expect the message to appear and wrap these commands in an if statement, e.g.

if !IsPseudoColor

	message Non PseudoColor display
	in warning_popup
	push warning.OK
endif
This same check will work whatever display hardware or window manager you are using.

The size of application dialogs is also important. Two dialogs shown simultaneously may both be fully visible on one display, overlap on another or be placed one on top of the other on a third. This can result in application-modal warning messages disappearing behind the main dialog, and your application apparently locking-up.

The following test script fragment demonstrates how to handle such a problem:

if !IsVisible(open_file_dialog)

	error The Open File dialog is off screen
endif
See "Display Expressions" on page 833 for more information on handling different display types.

If your application exhibits different behavior on different displays, your tests need to be written to accommodate this. For example, the application may put up a warning dialog to tell the user to expect some degradation of display quality.

Now consider the selection of an option from an option menu. While a standard script will certainly make the selection, a good testing script will check that the selection has been made.

The example below shows how we test that the Language option has been set to an expected value in the Sun WorkShop Visual Generate dialog:

if !languageOption->menuHistory:'cppButton'

		message FAIL: Language option error.
		printres languageOption->menuHistory
		message expected cppButton
	endif

What to do when a test fails

There are three ways to deal with a test failure:

stop the test but stay in the application
stop the test and exit from the application
abort the test and carry on with the next test in the test sequence
Each relates to a particular visu_replay command line flag:

-user-on-error - stays in the application
-exit-on-error - exits from the application
-skip-on-error - skips to the next test
The best way to handle failure is to prepare for it in your script. Use conditional sequences and take appropriate actions (e.g. output a message) when a failure occurs.

Another useful aid to the location of test failure is the -v command line flag. This displays commands from the script on standard out as they are executed. Once you have located the problem, you can create a smaller script to reproduce it. This can then be used (perhaps in conjunction with your favorite debugger) to identify the problem. It can also be added to your regression test suite to demonstrate that the bug has been fixed.


Using Testing Macros

We described in "Script Fragmentation" on page 453 how test scripts can be modularized using macros which define actions which are repeated (e.g. opening dialogs, starting the application, typing into a text field etc.) This makes the scripts easier to create, amend and check by hand.


Example of Scripts Using Macros

In order to illustrate how macros can be used to create modular scripts, an extract from the Sun WorkShop Visual test scripts is listed below as an example. This short script does the following:

  1. Starts Sun WorkShop Visual
  2. Creates a design containing a Shell and Form
  3. Gives the Form a variable name
  4. Saves the design, specifying a filename
  5. Exits Sun WorkShop Visual
To do this in a way which makes the top-level script more readable, we shall use the macro preprocessor, m4. This is available on all UNIX systems.

The high-level script to do the above is:

include(Defs.m4)

StartUp()
shell   date
Palette(xd_XmDialogShell)
Palette(xd_XmForm)
VariableName(myform)
SaveDesignAs(mydesign.xd)
message Test Sequence Over
shell   date
Finish()
Most of the above script consists of macro calls. See Appendix  A, "Sun WorkShop Visual Replay Command Syntax", starting on page 815 for more details on which part of the syntax are keywords.

The macro definition script, named Defs.m4, looks like this:

define(HandleExpectedWarning,

        in warning_popup
                push warning.OK)
define(StartUp,
        if !IsPseudoColor
                message Non PseudoColor display
                HandleExpectedWarning()
        endif)
define(Palette,
        in ApplicationShell
                push $1)
define(VariableName,
        in ApplicationShell
			multiclick nb_vn_t
			type $1
			key Return)
define(SaveDesignAs,
        in ApplicationShell
                cascade file_menu
                        select fm_menu.fm_saveas
                in save_dialog_popup
                        doubleclick Text
                type $1
			push save_dialog.OK)
define(Finish,
        in ApplicationShell
                cascade file_menu
                                select fm_menu.fm_exit
                if in save_changes_dialog
                        push xd_question.xd_question_cancel_b
                endif)
The following command:

m4 Test.in > Test.xds

creates the final script file which can be passed to Sun WorkShop Visual Replay. The file Test.in is the high-level script and Test.xds is the output file which will contain the final script with expanded macros.

Try out this example by typing in the files listed above and then using m4 to make the final script file. Having done this, run Sun WorkShop Visual Replay with Sun WorkShop Visual specifying Test.xds as the script to be replayed:

visu_replay -f Test.xds visu

You could take this example one step further by defining the names of the widgets on the Sun WorkShop Visual widget palette in a separate file and then defining the "Palette" macro so that it looks up the widget name from a high-level name such as "shell" or "form":

define(shell, xd_XmDialogShell)

define(form, xd_XmForm)
In this way the internal names are kept in one place where they can be maintained and changed more easily.


Debugging with Sun WorkShop Visual Replay

Running Sun WorkShop Visual Replay from the command line allows you to provide more than one application name if the application is an indirection.

For example, the following command:

visu_replay -f MyScript dbx AnApplication

would run a dbx session on the application AnApplication. Any debugger can be used. Using Sun WorkShop Visual Replay means that you can reach the stage at which you wish to start debugging quickly. To break into the debugger you can either reach the end of the script or place a "breakpoint" in the script. "breakpoint" is a keyword which is followed by the name of a widget. When the widget is activated, the application breaks into a debugger.


Extending the Sun WorkShop Visual Replay Widget Set


Overview

Sun WorkShop Visual Replay is based on the principle that the actions which are recorded in a script must be immediately recognizable as user actions.

Most actions which take place within a Motif application are described in terms of how a user interacts with its widgets (e.g. by clicking with one of the mouse buttons) or what is typed from the keyboard. This makes recording and replaying a Motif application very straightforward. It also makes it easy for a tester to understand, program and maintain scripts. The same must be true of any non-standard widget used in an application.

There are a number of Motif widgets (those for which the position in the widget is important) for which this approach does not immediately work. For most, e.g. Scales, ScrollBars, etc., a single mechanism will work for all instances of that widget. In a DrawingArea, or other custom widget, each instance of a widget may behave quite differently.

For example, although the recording software may observe a click in a drawing area, the user sees this action quite differently. He is interacting with objects that have been drawn in the drawing area by the application. But these objects only appear within the code of the application - they are not part of the interface.

Since you, as an application programmer, will know exactly what a click in a particular custom widget or drawing area actually means, you can easily provide routines which describe these actions so that they can be understood and used by people who wish to record and replay your widget.

If you are programming a drawing-area, or some other customized widget, you will already have written code to convert from an event at a particular (x,y) coordinate in that widget to a particular action in the application. Sun WorkShop Visual Replay provides interfaces which allow you to register converters that allow testers to make use of your routines.

You need to provide Sun WorkShop Visual Replay with two converters: one for recording, which converts an event at a particular (x,y) coordinate into an action and another, for replaying which converts that action to an (x,y) coordinate.

The conversion routine allows you to map the (x,y) coordinate to something which makes sense both to the user and to the widget itself.


Note - Some widgets provide an easy way to convert between (x,y) coordinates and the internal structure of the widget, e.g. the XmListYToPos function. This is the preferred method. Other widgets provide ways of determining and changing the state of a widget. For example you use XmScrollBarGetValues to record a user action on a scroll bar and XmScrollBarSetValues to replay that action. You can use the converters to program a widget directly if the event strategy is difficult to implement.
The same mechanism is used both for widget classes (e.g. third party widgets) and for custom widgets (e.g. the Motif XmDrawingArea widget).

The next two sections describe the converter routines. We then give an example which shows the creation of converters for the Motif XmList widget class.


Event to Name/Attribute Conversion Routine


Name

xdsXyToNameProc

-

interface definition for procedure used to
convert from an event to a name/attribute description


Synopsis

typedef int (*xdsXYToNameProc) (
	Widget widget,
	int    x,
	int    y,
	char**  name_p,
	char** attribute_p )

Inputs

widget

the widget that will use the routine

x

the x co-ordinate of the event

y

the y co-ordinate of the event

name_p

(return) pointer to a string that identifies the part of the widget

attribute_p

(return) pointer to a string that adds to the name, e.g. center, left, right


Usage

The routine should return 0 on failure, 1 on success. The strings that you assign to name_p and attribute_p are not freed by Sun WorkShop Visual Replay. Since copies are taken, you can use static storage.

If the routine fails, an error is reported.


Name/Attribute To Event Conversion Routine


Name

xdsNameToXYProc

-

interface definition for procedure used to
convert from a name/attribute description to an event


Synopsis

typedef int (*xdsNameToXyProc) (
	Widget widget,
	char*  name,
	char* attribute,
	int*  x_p,
	int*  y_p )

Inputs

widget

the widget that will use the routine

name

a string that identifies the part of the widget

attribute

string that adds to the name, e.g. center, left, right

x_p

(return) pointer to the x co-ordinate result

y_p

(return) pointer to the y co-ordinate result


Usage

The routine should return 0 on failure, 1 on success.


Notes

Sometimes it is very easy to program the effect you need to replay directly onto the widget, e.g. by setting a resource value or calling a convenience function, but extremely difficult to mimic the event sequence precisely. In these circumstances, you can handle it yourself in the routine.

You should still return success, but set the (x,y) co-ordinates to negative values. The standard mechanism simulates a single click within a widget and expects positive coordinates.


An Example

This worked example comes from the Sun WorkShop Visual Replay sources. It is an example of how to register converters for a class of widgets, in this case the Motif XmList widget class. It demonstrates how a click in an XmList widget can be converted to the selection of a particular instance of an element from that list. This is the actual mechanism used for XmList widgets by Sun WorkShop Visual Replay. An example of its use is illustrated in the script fragment below:

in my_shell

		push my_list_widget(`this line',1)
When the script is replayed, a button click is simulated at the appropriate (x,y) coordinates within the widget.

Once you have read through this example, you will be able to:

integrate a 3rd party widget
use the conversion mechanisms and understand the widget(name,attribute) notation
understand how we have implemented the XmList widget class
The source files for the example, together with a Makefile are provided in the $VISUROOT/src/examples/replay/cvtXm directory, where $VISUROOT is the location of your Sun WorkShop Visual installation.

The contents of this directory are listed below:

File
Description

Makefile

allows you to build the shared object on different platforms

README

shows how to build your shared object

motif*.c

conversion routines

register.c

registration routines

xds*

support files

The support files provide the framework which allows your shared object to communicate with the Sun WorkShop Visual Replay engine. You do not need to change any of these files.

For the purposes of this example, we will be examining motif2.c which contains the XmList converters and register.c which includes code to register these converters.

The example illustrates the three stages in extending the Sun WorkShop Visual Replay widget set:

  1. Event (x,y) to Name/Attribute pair - xdsXYToNameProc
  2. Name/Attribute pair to Event (x,y) - xdsNameToXYProc
  3. Registering Converters - xdsRegisterContextHandler
The three associated routines are:

  1. xdsListXyToName() - which takes an (x,y) position and returns a name/attribute pair.
  2. xdsListNameToXy() - which takes a name/attribute pair and returns an (x,y) position.
  3. xdsRegister() - which registers a widget and its associated converters with Sun WorkShop Visual Replay
What is important is the structure of the converter code and how the converters are registered and not how we have implemented the XmList converters.


xdsListXyToName()

This function is in motif2.c. It converts an (x,y) coordinate to a name/attribute pair, illustrating the xdsXyToNameProc interface definition structure. This function is used when Sun WorkShop Visual Replay is in record mode.

int

xdsListXyToName( widget, x, y, namep, attrp)
	Widget widget;
	int x, y;
	char ** namep;
	char ** attrp;
{
	extern Boolean XmStringCompare();




	extern char *  xdsCvtXmStringToString();
	extern Boolean xdsCvtSetListError();
	extern int     xdsCvtListFailure();
	extern Boolean xdsCvtGetXmListEntries();




	extern int XmListYToPos();




	static char name[255];
	static char count[20];




	int pos;
	int len = 0;
	int n;




	int instance = 1;
	XmString * list = (XmString*)0;
	XmString   item;




	/* get the element */
	if (!xdsGetXmListEntries1( widget,&list, &len)) {
		return xdsListFailure();
	}




	/* use XmListYToPos() to get the list element */
	pos = XmListYToPos2( widget, (Position)y);
	if (pos < 0 || pos > len) {
		xdsCvtSetListError(LIST_OUT_OF_BOUNDS);
		return xdsCvtListFailure();
	}




	item = list[--pos];
	for (n = 0; n < pos; n++) {
		if (XmStringCompare( item, list[n]) == True)
			instance++;
	}
	/* prepare the description */
	(void) sprintf ( count, "%d", instance);




	(void) strcpy  ( name, xdsXmStringToString3(item));




	*namep  = name;
	*attrp  = count;




	return 1;
}

xdsListNameToXy()

This function is also in motif2.c. It converts a name/attribute pair to an (x,y) coordinate, illustrating the xdsNameToXyProc interface definition structure. It is the complementary function to xdsListXyToName(). This function is used when Sun WorkShop Visual Replay is in replay mode

int

xdsListNameToXy( widget, name, attr, xp, yp)
	Widget widget;
	char * name;
	char * attr;
	int  * xp;
	int * yp;
{
	extern char * xdsCvtXmStringToString();
	extern Boolean xdsCvtSetListError();
	extern int    xdsCvtListFailure();
	extern int    xdsCvtSetListItem();
	extern Boolean xdsCvtGetXmListEntries();




	Position x, y;
	Dimension w, h;
	int pos;
	int len = 0;
	int n;
	char * s;
	int instance = 1;
	XmString * list = (XmString*)0;
	XmString   item;




	if ((instance = atoi(attr)) == 0) {
		xdsCvtSetListError(LIST_BAD_INSTANCE);
		return xdsCvtListFailure();
	}
	instance--;




	if (!xdsCvtGetXmListEntries( widget, &list, &len)) {
		xdsCvtSetListError(LIST_EMPTY_LIST);
		return xdsCvtListFailure();
	}




	for ( n = 0; n < len; n++) {
		s = xdsCvtXmStringToString(list[n]);
		if (strcmp( name, s) != 0)
			continue;
		if (instance--)
			continue;
		break;
	}




	if (n == len) {
		xdsCvtSetListError(LIST_ELEMENT_NOT_FOUND);
		return xdsCvtListFailure();
	}
	(void) xdsCvtSetListItem( widget, n+1);




	if (!XmListPosToBounds4( widget, n+1, &x, &y, &w, &h)) {
		xdsCvtSetListError(LIST_OUT_OF_BOUNDS);
		return xdsCvtListFailure();
	}




	*xp = x + (w/2);
	*yp = y + (h/2);
	return 1;
}

xdsRegister()

The function is called in register.c. It registers the two converters in motif2.c and those for the XmScrollBar, XmScale and XmDrawingArea widgets.

void

RegisterWidgets()
{
	extern Boolean xdsRegister();
	extern int xdsListNameToXy();
	extern int xdsListXyToName();
	extern int xdsScrollBarNameToXy();
	extern int xdsScrollBarXyToName();
	extern int xdsScaleNameToXy();
	extern int xdsScaleXyToName();
	extern int xdsDaNameToXy();
	extern int xdsDaXyToName();




	(void) xdsRegister( "XmList", xdsListNameToXy, xdsListXyToName);
	(void) xdsRegister( "XmScrollBar", xdsScrollBarNameToXy, 
xdsScrollBarXyToName);
	(void) xdsRegister( "XmScale", xdsScaleNameToXy, 
xdsScaleXyToName);
	(void) xdsRegister( "XmDrawingArea", xdsDaNameToXy, 
xdsDaXyToName);
}
void RegisterThisListWidget(
	Widget w;
{
	xdsRegisterContextHandler(w, xdsListNameToXy, xdsListXyToName);
}
The function is defined in xdsSetup.h and illustrates the xdsRegisterContextHandler interface definition structure.

Boolean

xdsRegister( classname, name2xy, xy2name)
	char * classname;
	int_f  name2xy;
	int_f  xy2name;
{
	bool_f bf = xdsGetRegisterFunction();




	if (!bf)
		return False;




	return (*bf)( classname, name2xy, xy2name);
}

Building the Example

The supplied Makefile is configured to build a shared object. A number of operating systems are supported. These can be listed by typing: make.

You only need to change the OBJECT line in the Makefile in order to build the shared object. This should be changed to:

OBJECT = cvt<classname>

where classname is the prefix of the widget class. In this example, the widget class is XmList, so we use the Xm prefix, i.e.

OBJECT=cvtXm

To create the shared object, type: make <system>. For example on a Solaris machine, you would type: make solaris. This would create a shared object called libcvtXm.so.

Once the shared object has been built, copy or link it into the directory $VISUROOT/lib/xds. It will then be loaded by Sun WorkShop Visual Replay when required.

The source files for registering converters, together with a Makefile are provided in the $VISUROOT/src/examples/replay/cvtTemplate directory, where $VISUROOT is the location of your Sun WorkShop Visual installation.


Adding converters for Customizable Widgets

The example described above relates to widget classes. For customizable widgets (i.e. a specific instance of a widget, such as a Motif XmDrawingArea) a mechanism for registering conversion routines is provided for you in the Sun WorkShop Visual distribution. This allows you to tailor the behavior of Sun WorkShop Visual Replay in order to allow it to record and replay user actions within individual instances of widgets (Motif or non-Motif).

The converter registration code is listed below:

int_f _xdsRegisterFunction = (int_f)0;

Boolean
xdsRegisterContextHandler( widget, name2xy, xy2name)
	Widget widget;
	int_f  name2xy;
	int_f  xy2name;
{
	if (!_xdsRegisterFunction)
		return False;
	return (*_xdsRegisterFunction)( widget, name2xy, xy2name, True);
}
A call must be made to this function in the application.


Note - The _xdsRegisterFunction function pointer variable is set to 0. This means that the routine will always return and do nothing in your application when it is run without Sun WorkShop Visual Replay. When you run with Sun WorkShop Visual Replay, the variable is set to point to the Register handler which then gets called.
The routine for registering converters is described below.


Registering Converters


Name

xdsRegisterContextHandler

-

interface definition for procedure used to
register a converter


Synopsis

Boolean xdsRegisterContextHandler(
	Widget widget,
	xdsNameToXYProc name2xy,
	xdsXYToNameProc xy2name)
Boolean xdsRemoveContextHandler( Widget widget)

Inputs

widget

the widget that will use the routines

name2xy

pointer to a conversion function for replay

xy2name

pointer to a conversion function for record


Description

This routine is for registering your own interpretations of events in a widget. You can call xdsRegisterContextHandler at any time after you have created the widget in your code, for example:

button1 = XmCreatePushButton ( shell1, "button1", al, ac );

	xdsRegisterContextHandler(shell1, func1, func2)
When replaying, Sun WorkShop Visual Replay will call the func1 routine. When recording, it will call the func2 routine.

The mechanism is available to you either as a source file (client.c), or as a precompiled library module (libxdsclient.a). In the former case, it has to be compiled with your application, in the latter case re-linked with it.

It has no impact on the application itself and can be left in it with no adverse effects.


Summary

To extend the Sun WorkShop Visual Replay widget set:

Create an (x,y) to name/attribute converter
Create a name/attribute to (x,y) converter
For widget classes:

Register the converters
Create a shared object and copy or link it to the $VISUROOT/lib/xds directory
For individual widgets:

Call xdsRegisterContextHandler to associate a widget with a specific function
Build your application with client.c or link it with the libxdsclient.a library
Use the contents of the supplied examples directory as a guide to writing, registering and building your converters.


Adding your own Sun WorkShop Visual Replay commands


Overview

The command set of Sun WorkShop Visual Replay is intended for replaying user actions and for checking the state of an application with respect to its widget hierarchy and its resource settings. You are not limited to this set of commands. You can extend it to include commands to meet your own needs, for example:

To produce screen dumps at various points in a replay session.
To do other sorts of consistency checking on the widget hierarchy - one example would be to interface with Doug Young's widgetlint library.
To insert a probe or a patch for a particular debugging problem. This will be of most use in a stripped optimized binary, where you do not have access to the full power of the debugger.
You have already seen in the previous chapter examples of user-defined modules which are loaded implicitly by the Sun WorkShop Visual Replay engine. You have also seen how to construct and build these modules.

import allows you to load a module of your own commands explicitly into a script. Once the module has been loaded the commands in it can be invoked using the user command. This section shows you how to produce such a module. The process is similar in many ways to that described in the preceding section.


An Example

We will describe how to create a module which contains one command. This command prints a message on standard error and the name of the current shell widget. You can use this example as a template for constructing your own commands.

The source files for the command, together with a Makefile are provided in the $VISUROOT/src/examples/replay/usertemplate directory, where $VISUROOT is the location of your Sun WorkShop Visual installation.

The contents of this directory are listed below:

File

Description

Makefile

allows you to build the extra command module on different platforms

README

shows how to build your command module

interface.c

contains the code for the extra command described in this section

xds*

support files

The support files provide the framework which allows your extra commands to communicate with the Sun WorkShop Visual Replay engine. You only need to change the xdsResources.h file - the remaining files prefixed with xds need not be altered in any way.

You only need to change the OBJECT line in the Makefile in order to build the module. Then build the module by typing:

make <systemname>

You then copy or link the shared object to the $VISUROOT/lib/xds directory:

The contents of the interface.c file are shown below:

#include <stdio.h>

#include <X11/Xos.h>
#include <X11/Xlib.h>
#include <X11/Intrinsic.h>
void

exampleHalloWorld( shell, message)
	Widget shell;
	char * message;
{
	if (!message)
		message = "no message";
	(void) fprintf ( stderr, "Widget %s says '%s'\n",XtName(shell), 
message);
}

As you can see, a user-defined function should have two arguments:

shell - is the current shell widget (passed in by Sun WorkShop Visual Replay)
message - a string
The message is all the text which follows the user command syntax on that line in the script. The example script fragment below shows how a command would be accessed and used:

import usertemplate

	in ApplicationShell
user HalloWorld I'm here

Here the message is "I'm here".


The Interface

The interface between all objects and the Sun WorkShop Visual Replay engine takes place using the standard Xt resource handling routines.


Note - If you would like more information on resource structures, you are advised to consult Chapter 10 in Volume Four of the "X Toolkit Intrinsics Programming Manual" published by O'Reilly and Associates, or any other comparable book.
An entry for the new command is added to the resource list in xdsResources.h, as shown below:

{

		"HalloWorld", XtCCallback, XtRPointer, 	sizeof(XtPointer), 
XtOffsetOf(data_t,HalloWorld), XtRImmediate, 
(XtPointer)exampleHalloWorld
}

Only three items are of significance within this code:

"HalloWorld" is the resource name
XtOffsetOf(data_t, HalloWorld) gives the offset to the relevant entry in the data structure
(XtPointer)exampleHalloWorld is a pointer to the address of the function
A pointer to that resource is added to the data structure within this file:

typedef struct {
	int       type;
	XtPointer setValues;
	XtPointer getValues;
	XtPointer engineSetValues;
	XtPointer engineGetValues;
	/*-----------------------*/
	XtPointer HalloWorld;
} data_t;
Entries above the line in the data structure are common to all Sun WorkShop Visual Replay objects.

The last thing to do in this file is to declare the function:

extern void exampleHalloWorld();


Building the Module

As in the preceding chapter, you only need to change the OBJECT line in the Makefile in order to build the module. For this example, we change it to:

OBJECT=usertemplate

and then build the module by typing:

make solaris

Finally, we copy or link the shared object we have built to the $VISUROOT/lib/xds directory:

cp libusertemplate.so $VISUROOT/lib/xds

That is all there is to it.


Summary

To add a new command to the Sun WorkShop Visual Replay command set:

  1. Add the associated function to the interface.c file.
  2. Add an entry to the resource list in xdsResources.h
  3. Add a function pointer to the data structure in xdsResources.h
  4. Add an extern declaration of the function in xdsResources.h
  5. If necessary, change the OBJECT line in the Makefile
  6. Build the module
  7. Copy or link it to the $VISUROOT/lib/xds directory
You can create as many modules as you wish and load them into a script at any time.


Allowing Your Applications to be Recorded and Replayed

To permit users to use Sun WorkShop Visual Replay to record and replay your application, you must do the following:

have the following line in your code:
  xdsAllowUserAccess()
link the application with the libxdsclient.a library

See "Sun WorkShop Visual Replay and Sun WorkShop Visual Capture" on page 810 for tips and hints about using Sun WorkShop Visual Replay.



1 xdsGetXmListEntries() - we are going to return the element in the list; this is a simple routine that fetches the elements of an XmList.

2 XmListYToPos() - the Motif convenience function XmListYToPos( widget, y) does all the conversion that we need. It takes the y-coordinate of the event, and returns its position in the list.

3 xdsXmStringToString() - a routine to convert an XmString to a String.

4 XmListPosToBounds() - the Motif convenience function, XmListPosToBounds(), gives us the window bounding-box of a particular item in the list. This can be used to work out likely (x,y) coordinates for a click on that element.


Previous Next Contents Generated Index Doc Set Home