Compiled code coverage with `--gcov` doesn't work with Meson

[rgommers]

For how this is supposed to work, see the PR description of gh-3021. It works with python runtests.py --gcov, but doesn't work with dev.py:

$ python dev.py --gcov
Removing previous GCOV .gcda files...
The Meson build system
Version: 0.62.0
Source dir: /home/rgommers/code/scipy
Build dir: /home/rgommers/code/scipy/build
Build type: native build
Project name: SciPy
Project version: 1.9.0.dev0

meson.build:1:0: ERROR: Executables created by c compiler /home/rgommers/anaconda3/envs/scipy-dev/bin/x86_64-conda-linux-gnu-cc --coverage are not runnable.

A full log can be found at /home/rgommers/code/scipy/build/meson-logs/meson-log.txt

I made no attempt to figure out why yet.

Full log:

$ cat build/meson-logs/meson-log.txt 
Build started at 2022-03-29T22:07:44.629711
Main binary: /home/rgommers/anaconda3/envs/scipy-dev/bin/python3.10
Build Options: -Dprefix=/home/rgommers/code/scipy/build-install
Python system: Linux
The Meson build system
Version: 0.62.0
Source dir: /home/rgommers/code/scipy
Build dir: /home/rgommers/code/scipy/build
Build type: native build
Project name: SciPy
Project version: 1.9.0.dev0
Sanity testing C compiler: /home/rgommers/anaconda3/envs/scipy-dev/bin/x86_64-conda-linux-gnu-cc --coverage
Is cross compiler: False.
Sanity check compiler command line: /home/rgommers/anaconda3/envs/scipy-dev/bin/x86_64-conda-linux-gnu-cc --coverage sanitycheckc.c -o sanitycheckc.exe -Wno-unused-function -Wno-conversion -Wno-misleading-indentation -Wno-incompatible-pointer-types -D_FILE_OFFSET_BITS=64 -pthread -B /home/rgommers/anaconda3/envs/scipy-dev/compiler_compat -shared -Wl,-rpath,/home/rgommers/anaconda3/envs/scipy-dev/lib -Wl,-rpath-link,/home/rgommers/anaconda3/envs/scipy-dev/lib -L/home/rgommers/anaconda3/envs/scipy-dev/lib -Wl,-rpath,/home/rgommers/anaconda3/envs/scipy-dev/lib -Wl,-rpath-link,/home/rgommers/anaconda3/envs/scipy-dev/lib -L/home/rgommers/anaconda3/envs/scipy-dev/lib --coverage
Sanity check compile stdout:

-----
Sanity check compile stderr:

-----
Running test binary command: /home/rgommers/code/scipy/build/meson-private/sanitycheckc.exe

meson.build:1:0: ERROR: Executables created by c compiler /home/rgommers/anaconda3/envs/scipy-dev/bin/x86_64-conda-linux-gnu-cc --coverage are not runnable.

@Smit-create is this something you could look at?

[rgommers]

Once that works, --lcov-html should work too (assuming lcov is installed - easiest on Linux).

[Smit-create]

Yeah, I'll look into it.

[eli-schwartz]

Why is the C compiler x86_64-conda-linux-gnu-cc --coverage? It might make more sense to use Meson's -Db_coverage=true.

(This also sets up Meson's own coverage report generation targets, but creating in-tree coverage data might require figuring out editable installs, so that's a problem for a different time, probably.)

[Smit-create]

It might make more sense to use Meson's -Db_coverage=true.

I followed the following steps:

1. $ meson setup build -Db_coverage=true --prefix=$PWD/build-install
2. $ ninja -C build

$ meson test
No tests defined.

$ meson compile coverage-html

ERROR: Can't invoke target `coverage-html`: target not found

[eli-schwartz]

Right, my suggestion was about adding the right compiler flags automatically since Meson knows how to do that. I didn't say you should use meson test.

(You already have a test harness that AFAIK is performing admirably, so there's no pressing need to replace it like there was a need to replace numpy.distutils)

I would just use meson's coverage option to ensure that compiler detection of coverage compiler flags is done, and at the right place, solely as a replacement for passing --coverage as the $CC or $CFLAGS. Ignore Meson's gcovr/lcov integration.

[eli-schwartz]

Particularly I don't think coverage should be in the $CC as that affects meson compiler.links() tests, the sanity check "does this compiler work to create valid executables" test, and similar. Those should not be compiled with coverage, and doing so might be what is causing your errors. I got some really weird warnings when I tested doing that myself -- it really didn't like creating multiple test outputs which clobbered each others' coverage data.

[Smit-create]

I think the following works:

1. Setting up meson build:

meson setup build --prefix=$PWD/build-install -Db_coverage=true

2. Building and Installing with dev.py:

python dev.py --gcov

3. Generating lcov

python dev.py --lcov-html

[rgommers]

Works with runtests.py:

$ python runtests.py --gcov
$ find build -name '*.gcno' -type f -exec gcov -pb {} +
....
File 'build/src.linux-x86_64-3.10/build/src.linux-x86_64-3.10/build/src.linux-x86_64-3.10/scipy/sparse/linalg/_isolve/iterative/fortranobject.c'
Lines executed:2.38% of 504
Branches executed:1.32% of 454
Taken at least once:0.66% of 454
Calls executed:2.78% of 108
Creating 'build#src.linux-x86_64-3.10#build#src.linux-x86_64-3.10#build#src.linux-x86_64-3.10#scipy#sparse#linalg#_isolve#iterative#fortranobject.c.gcov'

File 'build/src.linux-x86_64-3.10/build/src.linux-x86_64-3.10/build/src.linux-x86_64-3.10/scipy/sparse/linalg/_eigen/arpack/fortranobject.c'
Lines executed:9.13% of 504
Branches executed:7.93% of 454
Taken at least once:4.41% of 454
Calls executed:9.26% of 108
Creating 'build#src.linux-x86_64-3.10#build#src.linux-x86_64-3.10#build#src.linux-x86_64-3.10#scipy#sparse#linalg#_eigen#arpack#fortranobject.c.gcov'

[rgommers]

python runtests.py --lcov-html works as well, as noted on gh-15968:

Unclear if the results are correct, test coverage seems incomplete. I only had a brief look, but this should be a start to be able to fix things up.

[rgommers]

See also this short blog post for some explanation: https://emptysqua.re/blog/code-coverage-python-c-extensions/