Doc tests are not being run

[danieleades]

Describe the bug
Tarpaulin is failing to run doc tests in one of my projects

To Reproduce
project is here - https://github.com/danieleades/rtps

• clone
• run cargo tarpaulin
• observe that no doc tests are run and coverage is zero
• run cargo test
• observe that their are doc tests and that they run

you can also see the github action output here - https://github.com/danieleades/rtps/runs/3850610019?check_suite_focus=true

Expected behavior
doc tests should be run and produce non-zero coverage

[xd009642]

So doc test coverage isn't ran by default you need to run --doc. I'm not sure in the long term, if I want to change the default behaviour as most people don't track coverage of doctests. In the short term, this won't be changing as doctest coverage only works on nightly because it requires the unstable --persist-doctests flag: rust-lang/rust#56925 and I'd rather not confuse people with their coverage stats potentially changing dramatically when they switch between different toolchains

[danieleades]

that makes sense. thanks for the explanation

[danieleades]

hang on a tic, if I use the --doc flag, then only the doctests are run. if i use the --tests flag, then the doc tests are excluded. Is there a way to run all of the unit tests, integration tests, and doc tests at once?

[xd009642]

the test-type flags are collected into a list so you can do cargo tarpaulin --doc --tests

[danieleades]

the test-type flags are collected into a list so you can do cargo tarpaulin --doc --tests

Pretty sure I tried that, and it complained...
Will double-check when I get a chance

[xd009642]

I double checked it on the all_test_types test project in the repo and it worked as expected, so hopefully it will work for you 🤞

$ cargo tarpaulin --version
cargo-tarpaulin version: 0.18.2
$ cargo +nightly tarpaulin --doc --tests
Oct 10 12:01:33.749  INFO cargo_tarpaulin::config: Creating config
Oct 10 12:01:34.156  INFO cargo_tarpaulin: Running Tarpaulin
Oct 10 12:01:34.156  INFO cargo_tarpaulin: Building project
Oct 10 12:01:34.156  INFO cargo_tarpaulin::cargo: Cleaning project
   Compiling all_test_types v0.1.0 (/home/daniel/personal/tarpaulin/tests/data/all_test_types)
    Finished test [unoptimized + debuginfo] target(s) in 0.42s
   Doc-tests all_test_types
   Compiling all_test_types v0.1.0 (/home/daniel/personal/tarpaulin/tests/data/all_test_types)
    Finished test [unoptimized + debuginfo] target(s) in 0.39s
Oct 10 12:01:35.340  INFO cargo_tarpaulin::process_handling::linux: Launching test
Oct 10 12:01:35.340  INFO cargo_tarpaulin::process_handling: running /home/daniel/personal/tarpaulin/tests/data/all_test_types/target/doctests/src_only_doc_test_rs_2_0/rust_out
Oct 10 12:01:35.627  INFO cargo_tarpaulin::process_handling::linux: Launching test
Oct 10 12:01:35.627  INFO cargo_tarpaulin::process_handling: running /home/daniel/personal/tarpaulin/tests/data/all_test_types/target/debug/deps/all_test_types-bc1fd442f8ad3c26

running 1 test
test only_test::tests::test_it ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s

Oct 10 12:01:35.952  INFO cargo_tarpaulin::report: Coverage Results:
|| Tested/Total Lines:
|| src/only_bench.rs: 0/3 +0.00%
|| src/only_doc_test.rs: 3/3 +100.00%
|| src/only_example.rs: 0/3 -100.00%
|| src/only_test.rs: 5/5 +0.00%
|| 
57.14% coverage, 8/14 lines covered, -9.523809523809518% change in coverage

[xd009642]

@danieleades did you have a chance to check this?

[danieleades]

@danieleades did you have a chance to check this?

I did! Works like a charm. Suspect I fat-fingered it on the first attempt.

Thankyou very much for the support