Skip to content

Commit

Permalink
contrib: add coverage-diff script
Browse files Browse the repository at this point in the history
We have coverage targets in our Makefile for using gcov to display line
coverage based on our test suite. The way I like to do it is to run:

    make coverage-test
    make coverage-report

This leaves the repo in a state where every X.c file that was covered has
an X.c.gcov file containing the coverage counts for every line, and "#####"
at every uncovered line.

There have been a few bugs in recent patches what would have been caught
if the test suite covered those blocks (including a few of mine). I want
to work towards a "sensible" amount of coverage on new topics. In my opinion,
this means that any logic should be covered, but the 'die()' blocks covering
very unlikely (or near-impossible) situations may not warrant coverage.

It is important to not measure the coverage of the codebase by what old code
is not covered. To help, I created the 'contrib/coverage-diff.sh' script.
After creating the coverage statistics at a version (say, 'topic') you can
then run

    contrib/coverage-diff.sh base topic

to see the lines added between 'base' and 'topic' that are not covered by the
test suite. The output uses 'git blame -s' format so you can find the commits
responsible and view the line numbers for quick access to the context, but
trims leading tabs in the file contents to reduce output width.

Signed-off-by: Derrick Stolee <dstolee@microsoft.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
  • Loading branch information
derrickstolee authored and gitster committed Oct 10, 2018
1 parent 1d4361b commit 783faed
Showing 1 changed file with 108 additions and 0 deletions.
108 changes: 108 additions & 0 deletions contrib/coverage-diff.sh
@@ -0,0 +1,108 @@
#!/bin/sh

# Usage: Run 'contrib/coverage-diff.sh <version1> <version2>' from source-root
# after running
#
# make coverage-test
# make coverage-report
#
# while checked out at <version2>. This script combines the *.gcov files
# generated by the 'make' commands above with 'git diff <version1> <version2>'
# to report new lines that are not covered by the test suite.

V1=$1
V2=$2

diff_lines () {
perl -e '
my $line_num;
while (<>) {
# Hunk header? Grab the beginning in postimage.
if (/^@@ -\d+(?:,\d+)? \+(\d+)(?:,\d+)? @@/) {
$line_num = $1;
next;
}
# Have we seen a hunk? Ignore "diff --git" etc.
next unless defined $line_num;
# Deleted line? Ignore.
if (/^-/) {
next;
}
# Show only the line number of added lines.
if (/^\+/) {
print "$line_num\n";
}
# Either common context or added line appear in
# the postimage. Count it.
$line_num++;
}
'
}

files=$(git diff --name-only "$V1" "$V2" -- \*.c)

# create empty file
>coverage-data.txt

for file in $files
do
git diff "$V1" "$V2" -- "$file" |
diff_lines |
sort >new_lines.txt

if ! test -s new_lines.txt
then
continue
fi

hash_file=$(echo $file | sed "s/\//\#/")

if ! test -s "$hash_file.gcov"
then
continue
fi

sed -ne '/#####:/{
s/ #####://
s/:.*//
s/ //g
p
}' "$hash_file.gcov" |
sort >uncovered_lines.txt

comm -12 uncovered_lines.txt new_lines.txt |
sed -e 's/$/\)/' |
sed -e 's/^/ /' >uncovered_new_lines.txt

grep -q '[^[:space:]]' <uncovered_new_lines.txt &&
echo $file >>coverage-data.txt &&
git blame -s "$V2" -- "$file" |
sed 's/\t//g' |
grep -f uncovered_new_lines.txt >>coverage-data.txt &&
echo >>coverage-data.txt

rm -f new_lines.txt uncovered_lines.txt uncovered_new_lines.txt
done

cat coverage-data.txt

echo "Commits introducing uncovered code:"

commit_list=$(cat coverage-data.txt |
grep -E '^[0-9a-f]{7,} ' |
awk '{print $1;}' |
sort |
uniq)

(
for commit in $commit_list
do
git log --no-decorate --pretty=format:'%an %h: %s' -1 $commit
echo
done
) | sort

rm coverage-data.txt

0 comments on commit 783faed

Please sign in to comment.