rustdoc --test ignores visibility

[ilammy]

Consider the following use case.

I'd expect rustdoc to compile and run doctests only for the public::function since that's the only thing exported from the example crate and thus accessible to rustdoc.

However, this is what I see (actual for current stable (1.4.0) and nightly (1.6, bac2b13) builds):

ilammy@ferocity ~/dev/rustdoc-visibility-issues
$ rustc --crate-type lib example.rs

ilammy@ferocity ~/dev/rustdoc-visibility-issues
$ rustdoc --test --library-path . example.rs

running 3 tests
test public::complex_helper_0 ... FAILED
test private::real_job_0 ... FAILED
test public::function_0 ... ok

failures:

---- public::complex_helper_0 stdout ----
        <anon>:2:16: 2:30 error: unresolved name `complex_helper` [E0425]
<anon>:2     assert_eq!(complex_helper(1, 2), 3);
                        ^~~~~~~~~~~~~~
<anon>:2:5: 2:41 note: in this expansion of assert_eq! (defined in <std macros>)

error: aborting due to previous error

---- private::real_job_0 stdout ----
        <anon>:3:9: 3:25 error: module `private` is private
<anon>:3     use example::private;
                 ^~~~~~~~~~~~~~~~
error: aborting due to previous error


failures:
    private::real_job_0
    public::complex_helper_0

test result: FAILED. 1 passed; 2 failed; 0 ignored; 0 measured

Rustdoc's intent, as I understand it, is documenting public API. Private items do not show up in docs (unless one explicitly disables the pass that strips them). I believe this should also include doctests.

It's natural to use the items being documented in doctest examples, but such examples for private items would not compile due to obvious reasons. I believe analyzing the code snippets is a bit overkill for rustdoc, but it's pretty reasonable to simply strip any doctests for private items (at least by default), assuming that they contain internal code examples and will not not compile outside of the crate being documented.

Not using documentation comments for private items or marking private code examples with ignore avoids the issue, but personally I think that these are poor workarounds. If private doctests are ignored by default then exporting a previously private item (together with its documentation) is simply a matter of adding pub to it (or to its module import in the crate). However, if any of these workarounds are used then one must explicitly convert regular comments to documentation comments and/or remove ignore from doctests in order to get everything right.

Some links to (somewhat) related issues:

• #15347 Add option to include private definition in the documentation
• #27104 Have rustdoc document private items, except use and extern crate

[QuietMisdreavus]

I've opened #54438 to filter out doctests on non-public items, but to also allow them to still be run with --document-private-items.

[QuietMisdreavus]

Based on the discussion in that PR, it seems like introducing such a change would be too controversial and will cause some existing doctests to stop being executed (by default). I do agree that such doctests are not useful to write, but introducing such a change is basically like breaking existing code - even though we're not causing something to fail that wasn't, we'd be stopping tests from executing that were being run before. Since that behavior has been in place all this time (the original issue was filed almost three years ago!), it's basically too entrenched to give up. I'm closing this issue in light of this.