Code coverage support

[sgammon]

Hey authors of rules_rust,

Has there been any consideration yet regarding source coverage with rules_rust? Using projects like cargo-cov, or cargo-tarpaulin, it looks pretty easy from the outside but perhaps not from within Bazel.

IIRC, instrumented_files= would need to be provided for Rust targets, but that part is relatively easy if there is already a tool that can produce lcov-compatible outputs.

[sgammon]

forget i said anything about crates: https://doc.rust-lang.org/nightly/unstable-book/compiler-flags/source-based-code-coverage.html

i've got a sample working locally outside of Bazel, which uses -Zinstrument-coverage/grcov/lcov to generate coverage data and merge with our canonical LCOV report, which already existed. it would be absolutely amazing if this was rolled into rules_rust, though.

i'm happy to contribute if lib maintainers can help me with a direction?

[UebelAndre]

Hey, I'm not super familiar with how coverage works but do think it should be something the rules support.

I'm wondering if other maintainers might be able to shine some light on this?

@hlopko @dfreese pinging you two because you seem to be pretty knowledgeable about some subtle details of Bazel and rustc. Hoping you'll have some thoughts 😄

[sluongng]

Good references should be from rules_go:

• Coverage of go test rules_go#140 (and some PRs mentioned here)
• https://docs.google.com/document/d/1-ZWHF-Q-qCKf19ik-t33ie58BkNurrYYzKR4OLtcilY/edit

Generally Bazel's coverage is not yet complete.
If your code coverage tooling does not follow the Java's convention, you gona have a bad time.

[hlopko]

CC @c-mita (I hear they're looking into coverage these days :)

[UebelAndre]

#1077 seems relates

[keith]

I've been looking into this a bit, and it's definitely possible today by using the C++ related coverage script attribute. It feels a bit like a hack but afaict there still no real movement in bazel around this.

[UebelAndre]

I've been looking into this a bit, and it's definitely possible today by using the C++ related coverage script attribute. It feels a bit like a hack but afaict there still no real movement in bazel around this.

@keith What's this coverage script attribute?

[keith]

Here's a super WIP version #1324

[hlopko]

FWIW I think that's our best shot at having working coverage for Rust anytime soon. Took a super-quick look and the PR looks reasonable. Thanks for working on it @keith!

[keith]

Thanks @hlopko !

[c-mita]

I've been looking into this a bit, and it's definitely possible today by using the C++ related coverage script attribute. It feels a bit like a hack but afaict there still no real movement in bazel around this.

I have a personal desire to fix this; to create a general interface for rules to use to support coverage collection (along with cross-language support). But other things have stolen my time.

The _lcov_merger and _collect_cc_coverage attributes are your best bet for now.
I suppose technically those form something of an API, but they're completely undocumented.

[keith]

Yea I haven't addressed _lcov_merger, but am using _collect_cc_coverage in #1324, should work fine for now

[UebelAndre]

Does #1324 address the concerns here?

[rbtcollins]

I've been poking at our bazel setup to see why we don't get coverage, and it seems the rules_rust coverage data is incompatible with --experimental_split_coverage_postprocessing - I'm not sure why this should be.

Should I file a separate bug for that, or is this bug being used as a tracking bug for coverage for rust in general?

[cristifalcas]

same for me, it looks like coverage is not created when I have the flag experimental_split_coverage_postprocessing enabled