Maintenance Programmer Skills – Testing

Surely one of the most critical skills for maintaining software is testing.  Why is testing important and how is it used in the process?

Testing is used for at least these purposes:

• In corrective maintenance (fixing production defects):
  • To reproduce and locate problems
  • To test fixed functionality
  • To regression test code that has been fixed
• In adaptive maintenance (enhancements)
  • To test new functionality
  • To regression test changed code

An important technique for any of these types of testing is the use of “test harnesses”, I believe first labeled as such in Working Effectively With Legacy Code, by Michael Feathers. (As an aside, let me say that while this is a very good book, it is heavily oriented to Java and other OO languages.  I found it difficult to apply the bulk of the book to the AS/400 – System i RPG world.)

A test harness is code that is written for the sole purpose of exercising and testing some piece of code that has been modified – most experienced programmers have used this method.  For example, let’s say we have PgmA calling PgmB.  It is PgmB that we want to change as it performs some calculations on the parameters passed to it and then returns some resulting parameters.  PgmB is not directly callable, it is only called by PgmA.  However, PgmA is a highly complex Order Entry program, and it only calls PgmB after the user (tester) enters data requiring a number of steps and complex test data setup.

Rather than enter test orders into PgmA to test PgmB, we write a test harness, PgmC, which reads test parameters from a little file we’ve set up, passes them to PgmB and then prints the results.  Using this test harness allows us to quickly run many different values through PgmB to check the results.  We can limit our execution of PgmA to just a few samples in order to prove that the interface is still intact.

Another key concept described in this book concerns error localization.  The principle states that the closer our testing is to the code we changed, the better the tests.  For example, let’s say we have a CL program that calls four RPG programs, one after the other, with each program outputting a file that is then read by the next (just to make an example.)  If we make a change in the first program we want to check the results of its output directly.  We do not want to rely only on checking the output of the fourth program.  In fact, of course, we want to test all possible outputs, but the principle here is that we want to check results as close to the change as possible, otherwise it is possible that we can receive both false positive and false negative results as a consequence of other, unknown errors in processing downstream from the changed code.  The more downstream code we run through before checking results, the greater the likelihood of false positive or negative results.

There are many aspects to testing for software maintenance, these are a couple – more to come.