Skip to content

Commit

Permalink
[clang][NFC] Update top-level Code Coverage documentation to include …
Browse files Browse the repository at this point in the history
…MC/DC.
  • Loading branch information
evodius96 committed Jan 22, 2024
1 parent 95c1039 commit 8789b7e
Show file tree
Hide file tree
Showing 2 changed files with 60 additions and 5 deletions.
52 changes: 47 additions & 5 deletions clang/docs/SourceBasedCodeCoverage.rst
Expand Up @@ -64,6 +64,11 @@ To compile code with coverage enabled, pass ``-fprofile-instr-generate
Note that linking together code with and without coverage instrumentation is
supported. Uninstrumented code simply won't be accounted for in reports.

To compile code with Modified Condition/Decision Coverage (MC/DC) enabled,
pass ``-fcoverage-mcdc`` in addition to the clang options specified above.
MC/DC is an advanced form of code coverage most applicable in the embedded
space.

Running the instrumented program
================================

Expand Down Expand Up @@ -211,6 +216,10 @@ region counts:
| 4| 1|}
------------------
If the application was instrumented for Modified Condition/Decision Coverage
(MC/DC) using the clang option ``-fcoverage-mcdc``, an MC/DC subview can be
enabled using ``--show-mcdc`` that will show detailed MC/DC information for
each complex condition boolean expression containing at most six conditions.

To generate a file-level summary of coverage statistics instead of a
line-oriented report, try:
Expand Down Expand Up @@ -259,7 +268,7 @@ the exported data at a high level in the llvm-cov source code.
Interpreting reports
====================

There are five statistics tracked in a coverage summary:
There are six statistics tracked in a coverage summary:

* Function coverage is the percentage of functions which have been executed at
least once. A function is considered to be executed if any of its
Expand Down Expand Up @@ -288,10 +297,28 @@ There are five statistics tracked in a coverage summary:
that is comprised of two individual conditions, each of which evaluates to
either true or false, producing four total branch outcomes.

Of these five statistics, function coverage is usually the least granular while
branch coverage is the most granular. 100% branch coverage for a function
implies 100% region coverage for a function. The project-wide totals for each
statistic are listed in the summary.
* Modified Condition/Decision Coverage (MC/DC) is the percentage of individual
branch conditions that have been shown to independently affect the decision
outcome of the boolean expression they comprise. This is accomplished using
the analysis of executed control flow through the expression (i.e. test
vectors) to show that as a condition's outcome is varied between "true" and
false", the decision's outcome also varies between "true" and false", while
the outcome of all other conditions is held fixed (or they are masked out as
unevaluatable, as happens in languages whose logical operators have
short-circuit semantics). MC/DC builds on top of branch coverage and
requires that all code blocks and all execution paths have been tested. This
statistic is hidden by default in reports, but it can be enabled via the
``-show-mcdc-summary`` option as long as code was also compiled using the
clang option ``-fcoverage-mcdc``.

* Boolean expressions that are only comprised of one condition (and therefore
have no logical operators) are not included in MC/DC analysis and are
trivially deducible using branch coverage.

Of these six statistics, function coverage is usually the least granular while
branch coverage (with MC/DC) is the most granular. 100% branch coverage for a
function implies 100% region coverage for a function. The project-wide totals
for each statistic are listed in the summary.

Format compatibility guarantees
===============================
Expand Down Expand Up @@ -453,6 +480,21 @@ Branch coverage is tied directly to branch-generating conditions in the source
code. Users should not see hidden branches that aren't actually tied to the
source code.

MC/DC Instrumentation
---------------------

When instrumenting for Modified Condition/Decision Coverage (MC/DC) using the
clang option ``-fcoverage-mcdc``, users are limited to at most **six** leaf-level
conditions in a boolean expression. A warning will be generated for boolean
expressions that contain more than six, and they will not be instrumented for
MC/DC.

Also, if a boolean expression is embedded in the nest of another boolean
expression but separated by a non-logical operator, this is also not supported.
For example, in ``x = (a && b && c && func(d && f))``, the ``d && f`` case
starts a new boolean expression that is separated from the other conditions by
the operator ``func()``. When this is encountered, a warning will be generated
and the boolean expression will not be instrumented.

Switch statements
-----------------
Expand Down
13 changes: 13 additions & 0 deletions llvm/docs/CoverageMappingFormat.rst
Expand Up @@ -146,6 +146,19 @@ There are several kinds of mapping regions:
<span>}</span>
</pre>`

* Decision regions associate multiple branch regions with a boolean
expression in the source code. This information also includes the number of
bitmap bytes needed to represent the expression's executed test vectors as
well as the total number of instrumentable branch conditions that comprise
the expression. Decision regions are used to visualize Modified
Condition/Decision Coverage (MC/DC) in *llvm-cov* for each boolean
expression. When decision regions are used, control flow IDs are assigned to
each associated branch region. One ID represents the current branch
condition, and two additional IDs represent the next branch condition in the
control flow given a true or false evaluation, respectively. This allows
*llvm-cov* to reconstruct the control flow around the conditions in order to
comprehend the full list of potential executable test vectors.

.. _source code range:

Source Range:
Expand Down

0 comments on commit 8789b7e

Please sign in to comment.