Maintenance Programmer Skills – Parallel Testing 3/3

Creating test harnesses if needed, or utilizing other drivers of test execution is important for executing the code being tested if it is not easily callable in a frequently repeated way.  One of my own maxims:  the easier it is to test, the more you will test, and the better results you will obtain. The more the manual overhead required to execute a test, the less the programmer will actually do so.  It is therefore, an important skill to be able to creatively find ways to initiate program execution at non-standard points and stop it at non-standard points in order exercise modified code, for those cases that require it.  A further skills is to be able to develop a test harness of some kind, or make use of other available system features in order to drive execution of the modified code.

To reuse an earlier example:

A test harness is code that is written for the sole purpose of exercising and testing some piece of code that has been modified.  For example, let’s say we have PgmA calling PgmB.  It is PgmB that we want to change as it performs some calculations on the parameters passed to it and then returns some resulting parameters.  PgmB is not directly callable, it is only called by PgmA.  However, PgmA is a highly complex Order Entry program, and it only calls PgmB after entering data that requires a number of steps and complex test data setup.

Rather than enter test orders into PgmA to test PgmB, we write a test harness, PgmC, which reads test parameters from a little file we’ve set up, passes them to PgmB and then prints the results.  Using this test harness allows us quickly run many different values through PgmB to check the results.  We can limit our execution of PgmA to just a few samples in order to prove that the interface is still intact.

Capturing test results may be as simple as having two sets of files being updated or having two sets of spool files generated.  In some cases the effectiveness of monitoring of database changes can be significantly improved by use journaling to ensure that all actual database changes are observed.  A maintenance programmer should be comfortable with setting up journaling and extracting data from the journals.

Comparing test results and distinguishing meaningful discrepancies from non-meaningful discrepancies is another critical skill.  Again, use of creative ideas to compare the two sets of parallel test data can be very effective to increase the volume and scope of tests.  Given two parallel sets of output database or spool files, the use of SQL, Query or quick ad hoc programs can be used to very quickly compare thousands of test results.  Part of the comparison process typically involves dealing with non-meaningful discrepancies – knowing how to identify and exclude those discrepancies and filter down to the meaningful discrepancies is an important aspect of obtaining large volumes or quality tests.