Output filenames for -Z instrument-coverage

[vmx]

I'm trying out the new -Z instrument-coverage support. One issue I ran into that by default the output is stored in a file called default.profraw. That file gets overridden on every new run.

When running cargo test that is a problem. It runs several executables (e.g. for the library tests, integration tests doc tests), where each of them overrides the default.profraw.

I don't know if that is possible, but it would b great to have .profraw files named after the executable they were run with.

Until then what you can do is running cargo test with LLVM_PROFILE_FILE set to %m.profraw:

LLVM_PROFILE_FILE="%m.profraw" CARGO_INCREMENTAL=0 RUSTFLAGS="-Clink-dead-code -Copt-level=0 -Ccodegen-units=1 -Zpanic_abort_tests -Cpanic=abort -Coverflow-checks=off -Zinstrument-coverage -Cinline-threshold=0" cargo +nightly test

Which will then result in file names something like 52458732642933725367_0.profraw, which can then be merged into a single indexed profile file via llvm-profdata merge -o default.profdata *.profraw.

[richkadel]

Since the file is generated by your own target binary (not from a Rust language tool), it's not something the compiler can easily work around.

The default name default.profraw and handling of LLVM_PROFILE_FILE is embedded in the LLVM compiler-rt library, and it is not language specific (it works the same for Clang and Swift, for example). It may be more effective to file a feature request with the LLVM project.

[vmx]

@richkadel cargo test runs several binaries, couldn't cargo test then set LLVM_PROFILE_FILE to something sensible per test run by default and use LLVM_PROFILE_FILE if set by the user?

[richkadel]

That's a good point. All of the work to-date has been on rustc, so I haven't made any changes to cargo and that doesn't normally come to mind. Thanks for the suggested feature!

[vmx]

After playing more with the coverage stuff. I think it would be great if the coverage output could be placed next to the binary it was run with. For example if a test executable is at target/debug/deps/my_test-246fe4d9493aa502, the coverage output could be at target/debug/deps/my_test-246fe4d9493aa502.profraw.

This is useful as the llvm-cov expects to provide a binary with the coverage information in order to produce the report, this way you could easily map the profiling output to the corresponding binary.

[richkadel]

@vmx - Sounds good. I don't know if you'd be interested in working on this feature yourself, but I'd welcome the contribution.

I'm not likely to have time to work on this myself due to shifting work priorities.

If you can and want to submit a PR, please @richkadel mention me for a review (I will send feedback, but I'm not authorized to approve or anything), and add @tmandry and @wesleywiser as well. Either one of them can approve, once ready.

Thanks!

[scole66]

Might be worth a comment in the document about this problem. I've had -Zinstrument-coverage just bundled into my environment, and I just realized that my coverage broke because I added "cargo install cargo-binutils", which compiled a bunch of things, all with coverage enabled (oops), and every time I run "cargo cov ..." I destroy my profraw files, unwittingly.

[taiki-e]

@rustbot label -requires-nightly

(instrument-coverage has been stabilized)