All Manuals > LispWorks User Guide and Reference Manual > 10 Code Coverage


10.7 Understanding the code coverage output

In general there are five colors that are used to color the source code. Below these are named by the keyword argument to code-coverage-set-html-background-colors that changes them for the HTML coloring, and the default color is shown in parentheses:

fully-covered (Green)

Forms where every part was covered, that is all of the source subforms were executed.

partially-covered (GreenYellow)

Forms that are partially covered in a visible way, that is some of the source subforms were not executed.

hidden-partial (Orange)

Forms that are partially covered in a non-visible way, that is all the source subforms were executed, but some of them were expanded to forms that were not completely executed.

uncovered (Pink)

Uncovered forms, that is forms that were never executed.

eliminated (DeepPink)

Forms that were completely eliminated by the compiler.

Note that by default only hidden-partial, uncovered and eliminated are shown, so all colored forms indicate something that was not covered. However, inside a hidden-partial form the fully-covered subforms are always colored, regardless of the setting of fully-covered.

When counters are displayed, they have their own background color counters (MediumAquamarine), except for negative counters that use color counters-negative (Gold). In addition the colors error (Red) and warn (Yellow) are used when adding error or warning messages.

10.7.1 Eliminated forms

The compiler eliminates forms that it determines are not needed. These include:

To deduce that code does not have side effects, the compiler needs to know that the function calls in it have no side effects. The only function calls that the compiler knows to be free of side effects are either system functions or automatically defined functions such as structure accessors.

Unreachable code can happen explicitly but it can also happen implicitly, when the compiler eliminates it. The commonest case is when the compiler uses type inference to infer what a predicate will return. For example, suppose you have this definition:

(defun my-func (arg)
  (let ((sum (+ arg 10)))
    (if (consp sum)
        (car sum)
      (* sum 7))))

Since sum is a result of the call to + it must be a number and therefore cannot be a cons. Hence the compiler can infer that the consp call will always return nil, replace the call to consp by nil, eliminate the call to car, and go straight to the * call. In the coloring, the call to car will be shown as eliminated, while the if form will be shown as fully-covered. That is a little counter-intuitive, because the if and the consp are not actually in the compiled code, but they were effectively evaluated by the compiler at compile time.

10.7.2 Displaying counters

By default, code coloring adds counters, indicating how many times a point in the code has been executed. Note that counts may also be negative, when the code coverage data that is displayed is the difference between datas (as generated by functions like subtract-code-coverage-data). When a count is 0 the counter is not displayed.

By default, if the counter for a subform is the same as the counter of the parent form, the counter for that subform is not displayed. Thus when you see a subform without a counter, it means it was executed the same number of times as its parent. The counter is displayed even if it is the same in situations when the code coverage status of the forms is different, for example if the parent is partially-covered but the subform is fully-covered.

It is possible to force all counters to be displayed, by passing :show-counters :all to code-coverage-data-generate-coloring-html and the other displaying functions. Note that the compiler does not generate counters for forms that it can deduce will be always executed the same number of times as the parent, so for these it will never display counters.

10.7.3 Function forms where the function is not actually called

Entering a function form does not necessarily call the function, because during evaluation of the arguments there may be an exit out of the form. This exit can be local or non-local, and in general the compiler cannot tell whether it will happen or not, though it can be sure that it will not happen for specific cases like local variable references and self-evaluating forms. Therefore the compiler adds a counter just before the function is called. It is not obvious how to display this counter, and when it is 0 what part of the form has not been executed.

The current coloring regards the function name as the part of the form that is "being evaluated" when the function is actually called. Therefore it inserts the counter before the function name, and if it is 0 it colors the function name in the uncovered color. That means that in some cases you have a counter before the opening parenthesis that counts the times that the form was entered, and a counter after the opening parenthesis that counts the times that the function was called. Since the coloring normally skips counters that are inside a form where they have the same count as the counter of the form, you will see both numbers only when they are different.

10.7.4 Partially hidden

Partially hidden forms are forms where all the source code as shown was executed, but some macro expansion of it contains a conditional where one of the branches was not executed. Thus there is not any specific form to color as uncovered, but some part of the actual code was not executed. These forms are colored by a special color hidden-partial, which is Orange by default.

For example, if you have this code:

(defmacro did-it-p ()
  '(if *done* nil t))
(defun go-next-p ()

and whenever go-next-p is called *done* is true, then the false branch of the if form was not executed, but inside the definition of go-next-p there is no source that was not executed. Thus the form (did-it-p) is partially hidden.

Partially hidden forms can have subforms that are fully covered. These forms are colored by the color fully-covered even if fully covered forms are not colored otherwise (the default behavior), to clarify that they were fully covered.

LispWorks User Guide and Reference Manual - 13 Feb 2015