Overcover

This repository contains Overcover. Overcover is a testing tool meant to work in conjunction with the coverage profile data generated by the go test tool with the -coverprofile option; it computes an overall coverage of the entire source base and emits that information. Overcover can also optionally enforce a minimum coverage threshold, allowing test jobs to be failed if the testing coverage has been weakened.

Basic Usage

To use Overcover, install and execute it like so:

% overcover --coverprofile coverage.out

This will read the indicated coverage profile and emit a message describing the overall coverage represented by the profile. A threshold can be enforced, which will cause Overcover to exit with a status code of 1 if the coverage does not reach the threshold:

% overcover --coverprofile coverage.out --threshold 75

These values can also be passed to Overcover using environment variables named after the long option names--e.g., OVERCOVER_COVERPROFILE specifies a value for --coverprofile, and OVERCOVER_THRESHOLD specifies a value for --threshold.

The go test command does not include packages that do not contain any test files in the output coverage profile. To ensure that all code is counted in the coverage determination, Overcover can be provided the package specification in the same format taken by go test; it will count the number of statements and include that data in its calculation. For example:

% overcover --coverprofile coverage.out ./...

Build arguments may be provided to influence the selection of source packages; these are standard build arguments to the go tools, and are specified with a --build-arg prefix; the corresponding environment variable, OVERCOVER_BUILD_ARG, should contain a space-separated sequence of build arguments.

Finally, summary information on the coverage per package can be emitted using --summary (OVERCOVER_SUMMARY), and detailed per-file information can be emitted using --detailed (OVERCOVER_DETAILED); these two options may be used together, in which case the package data will be emitted prior to the file data. The output is sorted first to place the lowest coverage at the top, then it is sorted lexically by package or file name.

Configuration File

Overcover can also read a configuration file containing the target threshold, which may be in any number of formats as long as the file has a proper extension; e.g., for a YAML file, the configuration file name should end with ".yaml" or ".yml". The values in this file are named the same as the command line options, and the configuration file may be specified to Overcover using either OVERCOVER_CONFIG or the --config command line option:

% overcover --config config.yaml

An example of that file could be as follows:

---
threshold: 75

Automatically Updating the Threshold

When using a configuration file with Overcover, the Overcover tool can automatically update the threshold in the file, to help ensure that coverage can never decrease. This is done using the headroom options, --min-headroom and --max-headroom (OVERCOVER_MIN_HEADROOM and OVERCOVER_MAX_HEADROOM for environment variables, and min_headroom and max_headroom in the configuration files). If the total coverage exceeds the threshold plus the maximum headroom, then the threshold will be updated to be the total coverage minus the minimum headroom. An example of a configuration file set up to use this automatic update feature would be as follows:

---
threshold: 75
min_headroom: 1
max_headroom: 2

To provide a concrete example of the automatic threshold update, let's assume that we're using the above configuration file, and that the actual coverage is 80%. Since max_headroom is set to 2, that means that the coverage would be greater than threshold + max_headroom (77%), so the threshold would be updated to the coverage minus the min_headroom, or 79%, and this configuration file would become:

---
threshold: 79
min_headroom: 1
max_headroom: 2

(Note that the configuration file after the update likely won't look exactly like this, as it is programmatically generated.)

Automatic threshold update only occurs when a configuration file is specified using --config or the environment variable OVERCOVER_CONFIG, and when max_headroom is set to a value strictly greater than min_headroom. Defaults for all these values are shown below.

Preventing Automatic Update

Sometimes it is convenient to prevent an automatic update of the configuration file, such as during a CI run. This can be done by using the --readonly flag, or setting the OVERCOVER_READONLY environment variable. (The variable need not have any particular value; it need only appear in the environment.) In this scenario, Overcover will exit with a status code of 5 to indicate that the configuration file needs to be updated with a new threshold; it will also emit a human-readable error message to the standard error stream indicating what the threshold should be set to.

Options/Configuration Table

Configuration	Environment Variable	Command Line Option	Default	Description
	OVERCOVER_COVERPROFILE	--coverprofile (-p)	Required	Name of the coverage profile generated by go test.
threshold	OVERCOVER_THRESHOLD	--threshold (-t)	0.0	Minimum coverage threshold required.
min_headroom	OVERCOVER_MIN_HEADROOM	--min-headroom (-m)	0.0	Minimum headroom. Used to compute a new threshold.
max_headroom	OVERCOVER_MAX_HEADROOM	--max-headroom (-M)	0.0	Maximum headroom. Used to determine when the threshold must be updated.
	OVERCOVER_CONFIG	--config (-c)	None	Specifies the name of the configuration file to use.
	OVERCOVER_READONLY	--readonly (-r)		Specifies that the configuration file should not be updated.
	OVERCOVER_BUILD_ARG	--build-arg (-b)	None	Specifies a build argument for package selection.
	OVERCOVER_SUMMARY	--summary (-s)		Specifies that per-package summary information should be emitted.
	OVERCOVER_DETAILED	--detailed (-d)		Specifies that per-file coverage information should be emitted.
		--help (-h)		Emits help text describing how to use Overcover.

Key:

• Required - Option must be provided.
• None - No default, not required.