coverage: lcov result incorrect #6747
Description
Describe the bug
With lcov, the coverage report is correct when src and test code are in different directories, and the test compiles the code in src directly.
The issue could be described in https://stackoverflow.com/questions/27556872/lcov-doesnt-match-gcov-coverage-results-for-a-googletest-unit-test
To Reproduce
With below-simplified code:
$ tree .
.
├── meson.build
├── src
│ ├── foo.cpp
│ ├── foo.hpp
│ ├── main.cpp
│ └── meson.build
└── test
├── meson.build
└── test_foo.cpp
$ find ./ -type f | xargs tail -n +1
==> ./test/test_foo.cpp <==
#include <gtest/gtest.h>
#include "foo.hpp"
TEST(FOO, foo)
{
EXPECT_EQ(foo(1, 2), 3);
}
==> ./test/meson.build <==
gtest = dependency('gtest', main: true, disabler: true)
test_foo = executable(
'test_foo',
'../src/foo.cpp',
'test_foo.cpp',
include_directories: '../src',
dependencies: gtest)
test('foo', test_foo)
==> ./meson.build <==
project('foo', 'cpp',
version: '0.1', meson_version: '>=0.49.0',
default_options: [
'warning_level=3',
'werror=true',
'cpp_std=c++17',
])
subdir('src')
subdir('test')
==> ./src/meson.build <==
executable(
'foo',
'foo.cpp',
'main.cpp',
)
==> ./src/main.cpp <==
#include "foo.hpp"
#include <cstdio>
int main()
{
auto ret = foo(1, 2);
printf("%d\n", ret);
return 0;
}
==> ./src/foo.hpp <==
#pragma once
int foo(int a, int b);
==> ./src/foo.cpp <==
#include "foo.hpp"
int foo(int a, int b)
{
return a + b;
}
When both lcov and gcovr are available:
$ meson -Db_coverage=true build-with-lcov
$ ninja -C build-with-lcov test
$ ninja -C build-with-lcov coverage-html
$ lynx --dump build-with-lcov/meson-logs/coveragereport/index.html
LCOV - code coverage report
Current view: top level Hit Total Coverage
Test: Code coverage Lines: 3 9 33.3 %
Date: 2020-03-06 15:03:58 Functions: 4 6 66.7 %
Legend: Rating: low: < 75 % medium: >= 75 % high: >= 90 % Branches: 3
12 25.0 %
Directory Sort by name Line Coverage [1]Sort by line coverage Functions
[2]Sort by function coverage Branches [3]Sort by branch coverage
[4]src
0.0%
0.0 % 0 / 6 0.0 % 0 / 2 - 0 / 0
[5]test
100.0%
100.0 % 3 / 3 100.0 % 4 / 4 25.0 % 3 / 12
Generated by: [6]LCOV version 1.14-6-g40580cd
When the only gcovr is available:
$ meson -Db_coverage=true build-with-gcovr
$ ninja -C build-with-gcovr test
$ ninja -C build-with-gcovr coverage-html
$ lynx --dump build-with-gcovr/meson-logs/coveragereport/index.html
GCC Code Coverage Report
Directory: ./ Exec Total Coverage
Date: 2020-03-06 17:13:07 Lines: 5 9 55.6 %
Legend: low: < 75.0 % medium: >= 75.0 % high: >= 90.0 % Branches: 3 12
25.0 %
File Lines Branches
[1]src/foo.cpp
100.0 % 2 / 2 - % 0 / 0
[2]src/main.cpp
0.0 % 0 / 4 - % 0 / 0
[3]test/test_foo.cpp
100.0 % 3 / 3 25.0 % 3 / 12
Generated by: [4]GCOVR (Version 4.2)
We could see that lcov reports 0% in src, while gcovr reports 100% in src/foo.cpp.
gcovr is correct here.
Expected behavior
Lcov and gcovr should produce the same and correct result.
system parameters
Ubuntu 18.04 with GCC 9.2
Python 3.6.9
meson 0.52.0
ninja 1.9.0
lcov: 1.14-6-g40580cd (currently the master revision)
gcovr 4.2
Additional information:
The document from https://mesonbuild.com/Feature-autodetection.html says:
When doing a code coverage build, Meson will check for existence of the binaries gcovr, lcov and genhtml. If version 3.3 or higher of the first is found, targets called coverage-text, coverage-xml and coverage-html are generated. Alternatively, if the latter two are found, only the target coverage-html is generated. Coverage reports can then be produced simply by calling e.g. ninja coverage-xml. As a convenience, a high-level coverage target is also generated which will produce all 3 coverage report types, if possible.
However, when both lcov and gcovr are available, and running coverage-html target will prefer to use lcov than gcovr, which is a conflict with the document.
If meson could provide an option to specify the coverage tool, it could be much better.