Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Writing testable documentation examples for proc macros is not possible #58700

Closed
LukasKalbertodt opened this issue Feb 24, 2019 · 2 comments · Fixed by #62855
Closed

Writing testable documentation examples for proc macros is not possible #58700

LukasKalbertodt opened this issue Feb 24, 2019 · 2 comments · Fixed by #62855
Labels
A-doctests Area: Documentation tests, run by rustdoc A-macros Area: All kinds of macros (custom derive, macro_rules!, proc macros, ..) T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.

Comments

@LukasKalbertodt
Copy link
Member

LukasKalbertodt commented Feb 24, 2019

Suppose our main crate foo defines a trait Foo. Now we want to write a derive(Foo) proc macro (or any other kind of proc macro which emits code using Foo). We define that proc macro in the foo-macro crate. Our main crate foo wants to rexport the proc macro so that users only have to depend on foo.

The code for this example:

Cargo.toml

[package]
name = "foo"

[dependencies]
foo-macro = { path = "foo-macro" }

src/lib.rs

pub use foo_macro::Foo;
trait Foo {}

foo-macro/Cargo.toml

[package]
name = "foo-macro"

foo-macro/src/lib.rs

#[proc_macro_derive(Foo)]
fn derive_foo(_: TokenStream) -> TokenStream {
    // ...
    quote! { impl foo::Foo for #name {} }
}

The question is now: how to document the custom derive in a way such that cargo test can test included example code?. I don't think it's currently possible.

If we document the function derive_foo in the proc macro crate, we have to make all examples ignore because they cannot be compiled, because compiling them would require using the foo crate. But this leads to a cyclic dependency (not even dev-dependencies works). foo-macros cannot depend on foo.

But we also can't document the reexport (that documentation is ignored/not shown). So it's not possible to write the documentation in the foo crate either. As a consequence, we don't have checked documentation tests -- which probably leads to stale and incorrect example code.

Now I can think of three workarounds:

  • Add #[cfg(not(rustdoc))] to the reexport, and add some other item with #[cfg(rustdoc)] and the actual documentation to your foo crate. That way, the reexport doesn't happen when rustdoc generates the documentation and instead renders the other dummy item with the correct documentation. For function like proc macros, it might be fine to have a macro_rules macro as dummy item, but for other kinds of proc macros, this... absolutely not nice.
  • Add a feature to your main foo crate (like no_macro_reexport). When that feature is activated, your main crate does not reexport macros and does not depend on the foo-macro crate. Now the foo-macro crate can depend on foo via dev-dependencies and activate that no_macro_reexport feature. That prevents the cyclic dependency. But now your main crate has a feature that should be implementation detail and you often have to build your main crate twice.
  • Just don't bother and document your proc macros somewhere else (e.g. how serde does it). I don't think that's a good solution at all: rustdoc should be able to also properly document proc macros.

So neither of these workarounds is particularly nice. I guess it's clear that there should be some kind of solution to this.


Potentially related:

@jonas-schievink
Copy link
Contributor

Is #58696 also related? If rustdoc would properly inline the docs I'd expect doctests to also "just work".

I've also hit this issue with a custom derive I'm currently working on, but since those aren't really named items putting docs and examples on the derivable trait instead seems like the best solution for those.

@jonas-schievink jonas-schievink added the T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. label Feb 24, 2019
@LukasKalbertodt
Copy link
Member Author

@jonas-schievink Yes, good find, it's indeed very related. Although I'm not sure if a working #[doc(inline)] would completely solve this problem.

@jonas-schievink jonas-schievink added A-macros Area: All kinds of macros (custom derive, macro_rules!, proc macros, ..) A-doctests Area: Documentation tests, run by rustdoc labels Feb 24, 2019
Aaron1011 added a commit to Aaron1011/rust that referenced this issue Jul 27, 2019
Split off from rust-lang#62855

Currently, rustdoc ignores any doc comments found on 'pub use'
statements. As described in issue rust-lang#58700, this makes it impossible to
properly document procedural macros. Any doc comments must be written on
the procedural macro definition, which must occur in a dedicated
proc-macro crate. This means that any doc comments or doc tests cannot
reference items defined in re-exporting crate, despite the fact that
such items may be required to use the procedural macro.

To solve this issue, this commit allows doc comments to be written on
'pub use' statements. For consistency, this applies to *all* 'pub use'
statements, not just those importing procedural macros.

When inlining documentation, documentation on 'pub use' statements will
be prepended to the documentation of the inlined item. For example,
the following items:

```rust

mod other_mod {
    /// Doc comment from definition
    pub struct MyStruct;
}

/// Doc comment from 'pub use'
///
pub use other_mod::MyStruct;
```

will caues the documentation for the re-export of 'MyStruct' to be
rendered as:

```
Doc comment from 'pub use'
Doc comment from definition
```

Note the empty line in the 'pub use' doc comments - because doc comments
are concatenated as-is, this ensure that the doc comments on the
definition start on a new line.
bors added a commit that referenced this issue Aug 4, 2019
…laumeGomez

Use doc comments from 'pub use' statements

Split off from #62855

Currently, rustdoc ignores any doc comments found on 'pub use'
statements. As described in issue #58700, this makes it impossible to
properly document procedural macros. Any doc comments must be written on
the procedural macro definition, which must occur in a dedicated
proc-macro crate. This means that any doc comments or doc tests cannot
reference items defined in re-exporting crate, despite the fact that
such items may be required to use the procedural macro.

To solve this issue, this commit allows doc comments to be written on
'pub use' statements. For consistency, this applies to *all* 'pub use'
statements, not just those importing procedural macros.

When inlining documentation, documentation on 'pub use' statements will
be prepended to the documentation of the inlined item. For example,
the following items:

```rust

mod other_mod {
    /// Doc comment from definition
    pub struct MyStruct;
}

/// Doc comment from 'pub use'
///
pub use other_mod::MyStruct;
```

will caues the documentation for the re-export of 'MyStruct' to be
rendered as:

```
Doc comment from 'pub use'
Doc comment from definition
```

Note the empty line in the 'pub use' doc comments - because doc comments
are concatenated as-is, this ensure that the doc comments on the
definition start on a new line.
Centril added a commit to Centril/rust that referenced this issue Aug 24, 2019
…final, r=petrochenkov

Improve Rustdoc's handling of procedural macros

Fixes rust-lang#58700
Fixes rust-lang#58696
Fixes rust-lang#49553
Fixes rust-lang#52210

This commit removes the special rustdoc handling for proc macros, as we can now
retrieve their span and attributes just like any other item.

A new command-line option is added to rustdoc: `--crate-type`. This takes the same options as rustc's `--crate-type` option. However, all values other than `proc-macro` are treated the same. This allows Rustdoc to enable 'proc macro mode' when handling a proc macro crate.

In compiletest, a new 'rustdoc-flags' option is added. This allows us to
pass in the '--proc-macro-crate' flag in the absence of Cargo.

I've opened [an additional PR to Cargo](rust-lang/cargo#7159) to support passing in this flag.
These two PRS can be merged in any order - the Cargo changes will not
take effect until the 'cargo' submodule is updated in this repository.
Centril added a commit to Centril/rust that referenced this issue Aug 24, 2019
…final, r=petrochenkov

Improve Rustdoc's handling of procedural macros

Fixes rust-lang#58700
Fixes rust-lang#58696
Fixes rust-lang#49553
Fixes rust-lang#52210

This commit removes the special rustdoc handling for proc macros, as we can now
retrieve their span and attributes just like any other item.

A new command-line option is added to rustdoc: `--crate-type`. This takes the same options as rustc's `--crate-type` option. However, all values other than `proc-macro` are treated the same. This allows Rustdoc to enable 'proc macro mode' when handling a proc macro crate.

In compiletest, a new 'rustdoc-flags' option is added. This allows us to
pass in the '--proc-macro-crate' flag in the absence of Cargo.

I've opened [an additional PR to Cargo](rust-lang/cargo#7159) to support passing in this flag.
These two PRS can be merged in any order - the Cargo changes will not
take effect until the 'cargo' submodule is updated in this repository.
bors added a commit that referenced this issue Aug 29, 2019
…trochenkov

Improve Rustdoc's handling of procedural macros

Fixes #58700
Fixes #58696
Fixes #49553
Fixes #52210

This commit removes the special rustdoc handling for proc macros, as we can now
retrieve their span and attributes just like any other item.

A new command-line option is added to rustdoc: `--crate-type`. This takes the same options as rustc's `--crate-type` option. However, all values other than `proc-macro` are treated the same. This allows Rustdoc to enable 'proc macro mode' when handling a proc macro crate.

In compiletest, a new 'rustdoc-flags' option is added. This allows us to
pass in the '--proc-macro-crate' flag in the absence of Cargo.

I've opened [an additional PR to Cargo](rust-lang/cargo#7159) to support passing in this flag.
These two PRS can be merged in any order - the Cargo changes will not
take effect until the 'cargo' submodule is updated in this repository.
@bors bors closed this as completed in 1498608 Aug 29, 2019
LukasKalbertodt added a commit to LukasKalbertodt/lox that referenced this issue Sep 7, 2019
This moves documentation of derive macros from the trait to the
reexport. And it removes the dirty hack for `mesh!`. This was possible
because this issue was closed:
   rust-lang/rust#58700
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-doctests Area: Documentation tests, run by rustdoc A-macros Area: All kinds of macros (custom derive, macro_rules!, proc macros, ..) T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants