Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cfg(doctest) doesn't work as expected #67295

Open
GuillaumeGomez opened this issue Dec 14, 2019 · 16 comments
Open

cfg(doctest) doesn't work as expected #67295

GuillaumeGomez opened this issue Dec 14, 2019 · 16 comments
Assignees
Labels
A-doctests Area: Documentation tests, run by rustdoc C-bug Category: This is a bug. E-hard Call for participation: Experience needed to fix: Hard / a lot T-cargo Relevant to the cargo team, which will review and decide on the PR/issue. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.

Comments

@GuillaumeGomez
Copy link
Member

First, create empty lib crate.

cargo init --lib foo

With use crate::foo::bar:

//! Hello
//!
//! ```
//! use crate::foo::bar;
//! bar();
//! ```

#[cfg(doctest)]
pub fn bar ()
{
  println!("hello");
}
   Compiling foo v0.1.0 (/home/user/tmp/foo)
    Finished test [unoptimized + debuginfo] target(s) in 0.01s
     Running target/debug/deps/foo-d4d7a4ec5556181e

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

   Doc-tests foo

running 1 test
test src/lib.rs -  (line 3) ... FAILED

failures:

---- src/lib.rs -  (line 3) stdout ----
error[E0432]: unresolved import `crate::foo::bar`
 --> src/lib.rs:4:5
  |
4 | use crate::foo::bar;
  |     ^^^^^^^^^^^^^^^ no `bar` in the root

error: aborting due to previous error

For more information about this error, try `rustc --explain E0432`.
Couldn't compile the test.

failures:
    src/lib.rs -  (line 3)

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out

error: test failed, to rerun pass '--doc'

With use foo::bar:

//! Hello
//!
//! ```
//! use foo::bar;
//! bar();
//! ```

#[cfg(doctest)]
pub fn bar ()
{
  println!("hello");
}
   Compiling foo v0.1.0 (/home/user/tmp/foo)
    Finished test [unoptimized + debuginfo] target(s) in 0.24s
     Running target/debug/deps/foo-d4d7a4ec5556181e

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

   Doc-tests foo

running 1 test
test src/lib.rs -  (line 3) ... FAILED

failures:

---- src/lib.rs -  (line 3) stdout ----
error[E0432]: unresolved import `foo::bar`
 --> src/lib.rs:4:5
  |
4 | use foo::bar;
  |     ^^^^^^^^ no `bar` in the root

error: aborting due to previous error

For more information about this error, try `rustc --explain E0432`.
Couldn't compile the test.

failures:
    src/lib.rs -  (line 3)

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out

error: test failed, to rerun pass '--doc'
//! Hello
//!
//! ```
//! use crate::bar;
//! bar();
//! ```

#[cfg(doctest)]
pub fn bar ()
{
  println!("hello");
}

With use crate::bar:

   Compiling foo v0.1.0 (/home/user/tmp/foo)
    Finished test [unoptimized + debuginfo] target(s) in 0.23s
     Running target/debug/deps/foo-d4d7a4ec5556181e

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

   Doc-tests foo

running 1 test
test src/lib.rs -  (line 3) ... FAILED

failures:

---- src/lib.rs -  (line 3) stdout ----
error[E0432]: unresolved import `crate::bar`
 --> src/lib.rs:4:5
  |
3 | use crate::bar;
  |     ^^^^^^^^^^ no `bar` in the root

error: aborting due to previous error

For more information about this error, try `rustc --explain E0432`.
Couldn't compile the test.

failures:
    src/lib.rs -  (line 3)

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out

error: test failed, to rerun pass '--doc'
@GuillaumeGomez GuillaumeGomez added the T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. label Dec 14, 2019
@GuillaumeGomez GuillaumeGomez self-assigned this Dec 14, 2019
@jonas-schievink jonas-schievink added the C-bug Category: This is a bug. label Dec 14, 2019
@GuillaumeGomez
Copy link
Member Author

It seems to be way more complicated than I thought... Apparently, you need the code to be compiled in order to be able to link to it when running tests. However, this first compilation doesn't go through rustdoc at all and therefore, doctest isn't used. So unless I'm missing something, we're kinda stuck because it'd mean that we'd have to recompile the current lib by adding the doctest feature.

cc @rust-lang/compiler @rust-lang/cargo

@GuillaumeGomez
Copy link
Member Author

cc @sunjay (in case you saw something I missed?)

@sunjay
Copy link
Member

sunjay commented Dec 19, 2019

So unless I'm missing something, we're kinda stuck because it'd mean that we'd have to recompile the current lib by adding the doctest feature.

@GuillaumeGomez I'm not familiar with this stuff at all, but I guess we'd have to do something similar to what we do for #[cfg(test)] then? We would need to compile the crate with cfg(test) and then with cfg(doctest). Then we can run each with their respective versions of the crate. This seems like a reasonable approach. Is it difficult to implement?

My original issue just suggested setting cfg(test) during all tests (including doctests), which would be easier to implement, but probably not desirable given how much it would break things. It's probably best if we figure out a way to recompile the current lib with cfg(doctest).

@GuillaumeGomez
Copy link
Member Author

It needs to be done on cargo then. I had a discussion with @ehuss about this and they told me that it could be a big burden on performances. So To be debated...

@eclipseo
Copy link

The following is not working, I'm wondering if this is caused by the present bug of if I'm not using cfg(doctest) as intended?

mod hawktracer {
  cfg_if::cfg_if! {
    // Do not mix tracing and tests
    if #[cfg(all(feature="tracing", not(test), not(doctest)))] {
      pub use rust_hawktracer::*;
    } else {
      pub use noop_proc_macro::hawktracer;
    }
  }
}

@sunjay
Copy link
Member

sunjay commented Dec 22, 2019

@eclipseo That's because of this bug. The cfg currently only works while collecting tests to run. It does not affect compilation in the way you'd expect.

@GuillaumeGomez GuillaumeGomez added the T-cargo Relevant to the cargo team, which will review and decide on the PR/issue. label Jan 8, 2020
@GuillaumeGomez
Copy link
Member Author

Ping @rust-lang/cargo

@ehuss
Copy link
Contributor

ehuss commented Jan 10, 2020

I don't think we have any additional input to provide. We feel like the cost of rebuilding the library a third time is too high. Unfortunately I don't have any suggestions for alternative approaches.

@GuillaumeGomez
Copy link
Member Author

Could we make this rebuild a bit smarter and only perform it if we have a #[cfg(doctest)] somewhere? (and of course, not run it on the dependencies)

ObsidianMinor added a commit to ObsidianMinor/protrust that referenced this issue Jan 26, 2020
stepancheg added a commit to stepancheg/rust-protobuf that referenced this issue Mar 25, 2020
Unfortunately, `#[cfg(doctest)]` does not seem to work:
rust-lang/rust#67295
camshaft added a commit to aws/s2n-quic that referenced this issue Jul 3, 2020
doctests currently don't work well for conditional features
see: rust-lang/rust#67295
@mgeisler
Copy link
Contributor

Hi @GuillaumeGomez,

I just ran into this also — I wanted to add a helper function for my doctests but I couldn't get it to work at all. I was initially encouraged by your cfg(doctest) is stable and you should use it article, but I now understand that it's only talking about the test collection phase, not the actual test compilation phase.

Could you perhaps update the article with a warning about this (and a link to this bug)? I think that would be helpful for future readers :-)

@GuillaumeGomez
Copy link
Member Author

@mgeisler I updated the blog post.

@mgeisler
Copy link
Contributor

Thanks a lot!

@jyn514 jyn514 added the E-hard Call for participation: Experience needed to fix: Hard / a lot label Dec 15, 2020
dtolnay added a commit to serde-rs/serde that referenced this issue Jan 24, 2021
@Fishrock123
Copy link
Contributor

Fishrock123 commented Jan 29, 2021

I also ran into the need for this and reported it to zulip: https://rust-lang.zulipchat.com/#narrow/stream/122651-general/topic/.23.5Bcfg(test).5D.20within.20doc-tests

My usage is more-or-less to have the #[async_std::test] attribute also turn on when in doctests.

My specific usage is I would like cargo check to run for this example:
https://docs.rs/preroll/0.2.0/preroll/test_utils/fn.assert_json_error.html

Keep in mind I am trying to provide an accurate example of a test utility.

LLFourn added a commit to LLFourn/bdk that referenced this issue May 19, 2021
I wanted to only conditionally compile testutils but it's needed in
doctests which we can't conditionally compile for:

rust-lang/rust#67295
LLFourn added a commit to LLFourn/bdk that referenced this issue May 19, 2021
I wanted to only conditionally compile testutils but it's needed in
doctests which we can't conditionally compile for:

rust-lang/rust#67295
LLFourn added a commit to LLFourn/bdk that referenced this issue May 19, 2021
I wanted to only conditionally compile testutils but it's needed in
doctests which we can't conditionally compile for:

rust-lang/rust#67295
@domenicquirl
Copy link
Contributor

Hi all, I'd like to add that running into this is somewhat confusing if you don't happen to have read the aforementioned blog post. At least, it wasn't clear to me from this section in the rustdoc book that this doesn't work. Maybe this should be clarified if it's not on track to change anytime soon?

@armanriazi
Copy link

yes it doesn't work as expected

  1. I created crate check_doc_test.rs in lib module
#[cfg(doctest)]
pub mod check_doc_test {
    pub fn check_doc_macro(){
        doc_comment::doctest!("../README.md");
    }


    #[cfg(doctest)]
    pub fn bar ()
    {
    println!("hello");
    }
}
  1. Cargo.toml
[package]
edition = "2021"
name = "kno_lib"
version = "0.1.0"

[dev-dependencies]
doc-comment = "0.3.3"`
  1. cargo test --doc --workspace

I can not see any log test as "test result: ok."

  1. cargo doc --workspace --message-format short --no-deps --open --color always

` There is not any contents of REAMD.md on doc files in browser

  1. with apprericate. I am looking forward

@GuillaumeGomez
Copy link
Member Author

This is a cargo limitation unfortunately.

ebkalderon added a commit to ebkalderon/renderdoc-rs that referenced this issue Jan 27, 2023
This commit adds an unstable `ci` feature which reverts back the
previous non-compliant opening method because it enables tests to run in
CI without needing to painstakingly extract the test binary names from
`cargo build --tests --message-format json` and spawn each one in
`renderdoccmd capture $NAME`. Besides, this strategy is impossible
anyway because doctests compiled on-the-fly by `rustdoc` and do not have
external binaries with which to launch with RenderDoc. We also cannot
make the `RTLD_NOLOAD` conditional, i.e. `#[cfg(any(test, doctest))]`
and `#[cfg(not(any(test, doctest)))]`, due to:

rust-lang/rust#67295

As such, this internal crate feature will have to do.
ebkalderon added a commit to ebkalderon/renderdoc-rs that referenced this issue Jan 27, 2023
* Fix loading of RenderDoc library

It turns out this crate has been loading the RenderDoc API incorrectly:

> To do this you'll use your platforms dynamic library functions to see
> if the library is open already - e.g. `GetModuleHandle` on Windows, or
> or `dlopen` with the `RTLD_NOW | RTLD_NOLOAD` flags if available on
> *nix systems. On most platforms you can just search for the module
> name - `renderdoc.dll` on Windows, or `librenderdoc.so` on Linux, or
> `libVkLayer_GLES_RenderDoc.so` on Android should be sufficient here,
> so you don’t need to know the path to where RenderDoc is running from.
> This will vary by platform however so consult your platform’s OS
> documentation. Then you can use `GetProcAddress` or `dlsym` to fetch
> the `RENDERDOC_GetAPI` function using the typedef above.

This was reported long ago as an issue, and I'm addressing this now with
this commit.

It does change the runtime behavior of the library, though: the
application no longer attempts to load RenderDoc into memory if not
injected from the outside, so `RenderDoc::new()` will now return `Err`
when previously it would have succeeded.

* Restore integration and doctest support in CI

This commit adds an unstable `ci` feature which reverts back the
previous non-compliant opening method because it enables tests to run in
CI without needing to painstakingly extract the test binary names from
`cargo build --tests --message-format json` and spawn each one in
`renderdoccmd capture $NAME`. Besides, this strategy is impossible
anyway because doctests compiled on-the-fly by `rustdoc` and do not have
external binaries with which to launch with RenderDoc. We also cannot
make the `RTLD_NOLOAD` conditional, i.e. `#[cfg(any(test, doctest))]`
and `#[cfg(not(any(test, doctest)))]`, due to:

rust-lang/rust#67295

As such, this internal crate feature will have to do.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-doctests Area: Documentation tests, run by rustdoc C-bug Category: This is a bug. E-hard Call for participation: Experience needed to fix: Hard / a lot T-cargo Relevant to the cargo team, which will review and decide on the PR/issue. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests

10 participants