Maintenance Programmer Skills – Test Identification

How do you define appropriate tests when you have made a program change?  In maintenance programming there are two main flavors of such tests:

• Functional testing, to test that the code changes produce the desired result
• Regression testing, to test that everything else works as it did before the changes were made

A lot can be written about setting up systems for cataloging test cases, but that’s not my purpose here. My purpose is focused on defining desirable skills of maintenance programs for purposes of hiring and training.  So what should a good maintenance programmer know about defining appropriate tests?

For functional testing I believe the essential skill has to do with program slicing – both backward and forward.  This means the programmer needs to be able to identify all the paths that lead to the changed code, and all the paths that lead from it – and each for different reasons.

The paths that lead to the changed code are all the possible ways the code can come to be executed (backward program slicing).  Testing must be done to form the belief that the code will execute properly for all call paths.  It is not necessarily true that all such paths must be executed if it can be established that there is a subset of unique call path intersections.   Testing can then be restricted to that subset of paths.  The programmer should be able to do this analysis and design tests that will execute these paths.  Typically this involves analysis of upstream data requirements.

The paths that lead from the changed code comprise all the possible downstream code that could possibly be impacted by the change (forward program slicing.)  Testing must be done to form the belief that all these paths work correctly, and again it may be possible to identify a subset of paths that lead to confidence about the results.  A maintenance programmer should be able to do this type of analysis as well.

For regression testing there are two basic approaches the maintenance programmer should be familiar with:

• Identification and execution of established regression test cases
• Parallel testing

If the organization has a repository of established regression test cases then the maintenance programmer needs to know 1) how to identify the appropriate test cases, and 2) how to obtain a judgment about whether the existing regression test cases provide sufficient coverage.  The process of identifying appropriate test cases depends on the organization of the repository, but typically these are organized around business processes and use cases of some type.  I won’t attempt to define here how to judge the sufficiency of their coverage, but it is in a sense a matter of system-level slicing – forward and backward.

Parallel testing is a technique that all maintenance programmers should surely master.  There are a number of aspects and strategies to it, and I will write on that in my next post.