Planning for Change

Why Plan for Change?

Test Plan Architecture

Testing Project Selection Criteria

Testware Inventory

Change/Release Control

Version Management

Assign a Librarian

Challenges

Presented by Bob Johnston

ScriptTech, Inc.

Corte Madera, CA

Why Plan for Change?

Good serious use of AutoTester requires a substantial initial investment in training and development of automated tests. This investment pays large benefits when automated tests are repeated in regression testing and used in future releases of the software being tested. Planning for changes in your automated testing to match changes in your tested software is crucial to realize the long term benefits of using AutoTester. The "proof" that you've planned well for change will not come until future releases. Plan for change from the outset of your testing activities to protect your investment.

GOOD CHANGE MANAGEMENT IS CRUCIAL FOR LONG TERM BENEFIT

Test Plan Architecture

MAP YOUR TESTWARE DESIGN TO AN EXISTING STRUCTURE

How should you map tests to the real changing world? As your software goes through changes, it will be very important for you to determine which automated tests need to change to accommodate the changed software. To do this you must have some common "map" to connect the software structure to the testing structure. You may map your tests to programs, so if a particular program changes, you will know which tests need to change. You could also map using system menu trees, database structures or a maintained functional decomposition charts. Do this consciously from the start. Don't wait until the question comes up.

Worlds best System Chart Worlds best Test Plan

Once you have established a mapping, you can create charts to Cross Reference tests to system components such as programs.

Testing Project Selection Criteria

Three Criteria to make it easy and smart:

NOT ALL TESTING WILL BENEFIT FROM AutoTester.

SOME AUTOMATED TESTS WILL HAVE GREATER BENEFIT THAN OTHERS

CRITERIA FOR AUTOMATION BENEFIT:

FIND SUBSETS OF YOUR SYSTEM THAT MEET THE FOLLOWING:

1. EXPENSIVE OR IMPRACTICAL TO TEST MANUALLY

2. CRITICAL SYSTEM FUNCTIONS

3. STABLE FUNCTIONS OF SYSTEM THROUGH FUTURE RELEASES

FIND INTERSECTIONS OF 3 SUBSETS FOR MAXIMUM BENEFIT

Testware Inventory

The following should be included in your inventory:

all Autotester Scripts (Outlines, tapes, TXT files etc.)

AutoTester software programs and configuration files

Any host emulator programs and config files

AUTOEXEC.BAT, CONFIG.SYS, WIN.INI, etc. files

DOS/WIN/OS/2 Versions used during testing

documentation concerning test procedures or standards

complete description of hardware environment including keyboards, memory, display adapter, communications, etc.

All of these materials should be identified and treated as a set of "Testware".

Change/Release Control

TREAT YOUR TESTWARE AS YOU WOULD SOURCE CODE

Your test inventory should be treated just as you would treat the source code for your software. It should be included in any change/release control systems, backed up and archived just like program source code. The same organization or individual ultimately should be responsibility for both software source and Testware source materials even if the tests are developed and used by other groups.

Thousands of $$$$$

Version Management

EXPECT TO MAINTAIN 2-3 VERSIONS OF TESTWARE

Typically software exists in several different versions simultaneously (e.g. production, beta test, enhancement development). Obviously, you will have corresponding versions of your Testware. This greatly compounds the importance of good file management, naming conventions and test results logging. It makes sense to use the same version IDs for the Testware as the software itself. Your naming scheme and use of DOS directories must be able to support several versions of the tests at once. Make sure that documentation produced by your tests clearly identify which version of both the software and Testware are actually in use.

Don't forget, you may have development versions of your Testware as well. That is you may be developing test that are still being "tested". This can add another complexity to managing your Testware versions.

Assign a Librarian

Anyone with even a little experience knows that AutoTester

creates lots of DOS files, some of which must be kept in

careful sync (TAP & WND). A carefully thought out naming

convention is essential to managing Testware successfully.

Make sure that your naming convention "links" associated OTL,TAP, TXT and VAR files together. Keep in mind that you could be maintaining several different versions simultaneously. Use the full DOS file name, including the drive and directory in your naming scheme (e.g. E:\VER1.1\MDSTX10.OTL).

Develop a standard way to log test results. This will effect definition of standard variable usage, version control, as well as error reporting/tracking.

How to Start Right

DEVELOP STANDARDS AND PROCEDURES EARLY

Try it on a small scale first. We STRONGLY recommend that you do a small project ALL THE WAY THROUGH specifically to develop and verify all of the above considerations. A small Testware PILOT project will provide essential experience necessary to complete a large scale project successfully.

DESIGN

DEVELOP

CONSTRUCT

TEST

REVISE

RE-TEST

DOCUMENT