You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This tool has been extremely useful, thanks! By far the simplest to get working tool for the instrument-coverage (I had some trouble getting grcov to work)
One thing I think could be improved is the command line args - mainly about generating reports (where quite a lot of the arguments are mutually exclusive, e.g --no-run and any of the arguments like --lib, --html and --output-file`)
I think to steal some ideas from coverage.py could work, specifically to have a report subcommand.
Main downside of this is obviously a common "run tests and open report" usage becomes two commands, so instead of just cargo llvm-cov --html --open it would become cargo llvm-cov && cargo llvm-cov html --open.
However the upsides are:
the available options become smaller and more organised
maybe be a tiny bit more intuitive to use (e.g first few times I accidentally reran my testsuite because I assumed cargo llvm-cov --open would only open the report)
If gathering and reporting are two separate actions, it would be (if it's not already?) possible to gather coverage from multiple runs and report on them in one - e.g cargo llvm-cov test && cargo llvm-cov run --append-cov && cargo llvm-cov run --example blah --append-cov && cargo llvm-cov html which would run the tests and two executable, and generate a report of the combined coverage
I imagine the help output could look something like:
Example help output
$ cargo llvm-cov --help
cargo-llvm-cov
Cargo subcommand to easily use LLVM source-based code coverage (-Z instrument-coverage).
Use -h for short descriptions and --help for more details.
USAGE:
cargo llvm-cov [OPTIONS] [-- <ARGS>...] [SUBCOMMAND]
ARGS:
<ARGS>...
Arguments for the test binary
GLOBAL OPTIONS:
-h, --help
Print help information
-V, --version
Print version information
-v, --verbose
Use verbose output
Use -vv (-vvv) to propagate verbosity to cargo.
--color <WHEN>
Coloring [possible values: auto, always, never]
--manifest-path <PATH>
Path to Cargo.toml
--frozen
Require Cargo.lock and cache are up to date
--locked
Require Cargo.lock is up to date
--offline
Run without accessing the network
-Z <FLAG>...
Unstable (nightly-only) flags to Cargo
SUBCOMMANDS:
clean
Remove artifacts that cargo-llvm-cov has generated in the past
help
Print this message or the help of the given subcommand(s)
report
--ignore-filename-regex <PATTERN>
Skip source code files with file paths that match the given regular expression
--no-report
Run tests, but don't generate coverage report
--no-run
Generate coverage report without running tests
json <optional output file>
Export coverage data in "json" format
If output file is not specified, the report will be printed to stdout.
This internally calls `llvm-cov export -format=text`. See
<https://llvm.org/docs/CommandGuide/llvm-cov.html#llvm-cov-export> for more.
lcov <optional output file>
Export coverage data in "lcov" format
If output file is not specified, the report will be printed to stdout.
This internally calls `llvm-cov export -format=lcov`. See
<https://llvm.org/docs/CommandGuide/llvm-cov.html#llvm-cov-export> for more.
text <optional output directory>
Generate coverage report in "text" format
This internally calls `llvm-cov show -format=text`. See
<https://llvm.org/docs/CommandGuide/llvm-cov.html#llvm-cov-show> for more.
html <optional target directory>
Generate coverage report in "html" format
This internally calls `llvm-cov show -format=html`. See
<https://llvm.org/docs/CommandGuide/llvm-cov.html#llvm-cov-show> for more.
summary
Display only summary information for each file in the coverage data
run
--bin <NAME>...
Test only the specified binary
test
--doctests
Including doc tests (unstable)
This flag is unstable. See <https://github.com/taiki-e/cargo-llvm-cov/issues/2> for
more.
--no-fail-fast
Run all tests regardless of failure
-q, --quiet
Display one character per test instead of one line
--lib
Test only this package's library unit tests
--bins
Test all binaries
--example <NAME>...
Test only the specified example
--examples
Test all examples
--test <NAME>...
Test only the specified test target
--tests
Test all tests
--bench <NAME>...
Test only the specified bench target
--benches
Test all benches
--all-targets
Test all targets
--doc
Test only this library's documentation (unstable)
This flag is unstable because it automatically enables --doctests flag. See
<https://github.com/taiki-e/cargo-llvm-cov/issues/2> for more.
-p, --package <SPEC>...
Package to run tests for
--workspace
Test all packages in the workspace [aliases: all]
--exclude <SPEC>...
Exclude packages from the test
-j, --jobs <N>
Number of parallel jobs, defaults to # of CPUs
--release
Build artifacts in release mode, with optimizations
--profile <PROFILE-NAME>
Build artifacts with the specified profile
--features <FEATURES>...
Space or comma separated list of features to activate
--all-features
Activate all available features
--no-default-features
Do not activate the `default` feature
--target <TRIPLE>
Build for the target triple
When this option is used, coverage for proc-macro and build script will not be displayed
because cargo does not pass RUSTFLAGS to them.
The text was updated successfully, but these errors were encountered:
Main downside of this is obviously a common "run tests and open report" usage becomes two commands, so instead of just cargo llvm-cov --html --open it would become cargo llvm-cov && cargo llvm-cov html --open.
I think it would be okay to add a report subcommand for users who want to perform various actions on the report, but there is no need to break the existing one.
where quite a lot of the arguments are mutually exclusive, e.g --no-run and any of the arguments like --lib, --html and --output-file
Most of the mutual exclusivity issues should be fixed in #60
If gathering and reporting are two separate actions, it would be (if it's not already?) possible to gather coverage from multiple runs and report on them in one
This is already available. readme says:
You can merge the coverages generated under different test conditions by using --no-report and --no-run.
cargo llvm-cov clean --workspace # remove artifacts that may affect the coverage results
cargo llvm-cov --no-report --features a
cargo llvm-cov --no-report --features b
cargo llvm-cov --no-run --lcov
This tool has been extremely useful, thanks! By far the simplest to get working tool for the
instrument-coverage
(I had some trouble getting grcov to work)One thing I think could be improved is the command line args - mainly about generating reports (where quite a lot of the arguments are mutually exclusive, e.g
--no-run
and any of the arguments like--lib
,--html
and --output-file`)I think to steal some ideas from coverage.py could work, specifically to have a
report
subcommand.Main downside of this is obviously a common "run tests and open report" usage becomes two commands, so instead of just
cargo llvm-cov --html --open
it would becomecargo llvm-cov && cargo llvm-cov html --open
.However the upsides are:
cargo llvm-cov --open
would only open the report)cargo llvm-cov test && cargo llvm-cov run --append-cov && cargo llvm-cov run --example blah --append-cov && cargo llvm-cov html
which would run the tests and two executable, and generate a report of the combined coverageI imagine the help output could look something like:
Example help output
The text was updated successfully, but these errors were encountered: