How Do You Calculate an RPG Program’s Complexity?

I recently completed the initial implementation of some RPG code complexity measures in Codelyzer.  It has been shown for years on other platforms that having a systematic way of evaluating a program’s complexity facilitates improved planning, estimating, designing, testing, and system maintainability.  Based on the research I’ve done, I initially decided on four measures which seem to be the most widely recognized and referenced, and have been around for some time.  While I’ve seen no references to these measures for RPG, they have been widely used for COBOL, C, Pascal, Java, etc., and are considered to be language independent.  These are:

McCabe’s Cyclomatic Complexity – measures logic complexity by analyzing logic control statements such as IF, DO, etc.  This is generally done at a procedure or subroutine level.

Halstead’s Volume – measures operation and variable complexity.  Looks at the total number of operators and total number of variables used, and then also counts the unique instances of each.  The result is a log-based product of the total counts and unique counts.

Halstead’s Difficulty – Looks at the number of unique operators and the proportion of total variables to unique variables.

Maintainability Index – measures a program’s maintainability by using the McCabe and Halstead numbers, lines of code count and lines of comments count.  These numbers are combined with specific factors applied based on widespread experience.

There are numerous other measurements that have been devised over the years but I decided to start with these.

Initial results from analyzing a dozen or so programs is that this is interesting and revealing.  For example, in a particularly large and complex RPG program it was interesting to see how, out of 80 subroutines, the real complexity existed in about 10% of those subroutines.  The program is a much feared behemoth, but this analysis showed that the complexity lies in certain sections of the code.  Leading to the hope, that somehow, this could someday be improved.

I don’t know of any other efforts specifically meant to measure RPG complexity, but I would be interested in any other experiences.  Some of the facts about RPG seem to possibly require some adjustments to the formulas for these measurements.  A couple examples:

• The McCabe Cyclomatic Complexity formula does not count ELSE statements (it does count ELSEIF statements.)  However, based on the structure of RPG my sense is that ELSE statements are particularly confusing – at least with traditional, non-indented syntax.  I would almost count them as double.This formula also does not provide any adjustment for nesting.  My sense is that in traditional RPG syntax heavily nested logic is particularly onerous, especially when you mix in ELSE’s.

• The Maintainability Index counts comments as making the program more maintainable.  In my Codelyzer implementation I decided to only count comments that had a comment beyond the asterisk.  I did not count lines that were only “spacing” comments as comments.  In other languages these would typically just be blank lines and not “comments.”  Additionally, I wonder if comments that are actually just old, commented-out code should be counted.  These are in fact generally confusing rather than helpful, though of course, occasionally they can be helpful.  My sense was that these should not be counted as comments, but I didn’t see an immediate way to recognize them as being old code.

There are other possible shortcomings of these measurements for RPG, but these seem like a valuable start and I would be interested in the experience of others.  My belief is that good complexity measurements could be very useful for both planning and estimating purposes, and for system improvement projects.

Shown below is a sample screenshot from a not terribly complex RPG program.  There is one row per subroutine/procedure with three program summary rows at the top.  For the red and yellow warning colors I used industry standard threshold cut-offs.  I think these may need to be adjusted for RPG.

I will post the updated beta version of Codelyzer by the end of the week containing this new feature.