Use #[doc = include_str!()] in std #80534

LeSeulArtichaut · 2020-12-30T22:34:46Z

LeSeulArtichaut · 2020-12-30T22:39:16Z

Oh, nICE 🧊

petrochenkov · 2020-12-30T22:51:42Z

Interesting.
Could you make a minimized example? I'll look at it soon.

LeSeulArtichaut · 2020-12-30T23:09:45Z

So, uhhhhh, I can't reproduce the ICE locally. Instead, I get:

error: aborting due to 3392 previous errors; 6 warnings emitted

Some errors have detailed explanations: E0308, E0432, E0659.
For more information about an error, try `rustc --explain E0308`.
error: could not compile `core`

Which I think is both ridiculous and terrifying 👀

jyn514 · 2020-12-30T23:11:28Z

@LeSeulArtichaut make sure you rebased over the beta bump, it just happened an hour or so ago.

LeSeulArtichaut · 2020-12-30T23:20:11Z

Ah, yeah, that makes sense. But I still don't know why those errors were emitted. Rebasing...

LeSeulArtichaut · 2020-12-30T23:42:32Z

Backtrace:

thread 'rustc' panicked at 'non-eager expansion without a parent scope', compiler/rustc_resolve/src/macros.rs:238:22
stack backtrace:
   0: _rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::option::expect_failed
   3: rustc_resolve::macros::<impl rustc_expand::base::ResolverExpand for rustc_resolve::Resolver>::resolve_macro_invocation
   4: rustc_expand::expand::MacroExpander::fully_expand_fragment
   5: rustc_expand::expand::MacroExpander::expand_crate
   6: rustc_session::utils::<impl rustc_session::session::Session>::time
   7: rustc_interface::passes::configure_and_expand_inner
   8: rustc_interface::passes::configure_and_expand::{{closure}}
   9: rustc_data_structures::box_region::PinnedGenerator<I,A,R>::new
  10: rustc_interface::passes::configure_and_expand
  11: rustc_interface::queries::Queries::expansion
  12: rustc_interface::queries::<impl rustc_interface::interface::Compiler>::enter
  13: rustc_span::with_source_map
  14: rustc_interface::interface::create_compiler_and_run

LeSeulArtichaut · 2020-12-31T00:30:36Z

Minimized this and opened #80540. Also marked this PR as blocked on that issue.

LeSeulArtichaut · 2021-01-10T14:18:48Z

#80540 was fixed.

LeSeulArtichaut · 2021-01-10T14:24:55Z

Oh, of course #80563 isn't merged to the beta compiler... Should I close this and come back in 6 weeks? :)

jyn514 · 2021-01-10T14:31:39Z

@LeSeulArtichaut can you see if there are any other ICEs left so we don't have to wait another 6 weeks again after? You can check with x.py doc --stage 1 library/std I think.

LeSeulArtichaut · 2021-01-16T17:20:32Z

./x.py doc --stage 1 library/std --keep-stage-std 0 completes successfully.

jyn514 · 2021-02-06T22:33:45Z

@LeSeulArtichaut the version of nightly was bumped, but bootstrap hasn't switched to using the new (1.51) bootstrap compiler yet. That's a separate PR that usually happens closer to the stable release.

Use #[doc = include_str!()] in std cc rust-lang#78835 (comment) r? ``@jyn514``

Use #[doc = include_str!()] in std cc rust-lang#78835 (comment) r? ```@jyn514```

Use #[doc = include_str!()] in std cc rust-lang#78835 (comment) r? ````@jyn514````

Rollup of 16 pull requests Successful merges: - rust-lang#75807 (Convert core/num/mod.rs to intra-doc links) - rust-lang#80534 (Use #[doc = include_str!()] in std) - rust-lang#80553 (Add an impl of Error on `Arc<impl Error>`.) - rust-lang#81167 (Make ptr::write const) - rust-lang#81575 (rustdoc: Name fields of `ResolutionFailure::WrongNamespace`) - rust-lang#81713 (Account for associated consts in the "unstable assoc item name colission" lint) - rust-lang#82078 (Make char and u8 methods const) - rust-lang#82087 (Fix ICE caused by suggestion with no code substitutions) - rust-lang#82090 (Do not consider using a semicolon inside of a different-crate macro) - rust-lang#82213 (Slices for vecs) - rust-lang#82214 (Remove redundant to_string calls) - rust-lang#82220 (fix the false 'defined here' messages) - rust-lang#82313 (Update normalize.css to 8.0.1) - rust-lang#82321 (AST: Remove some unnecessary boxes) - rust-lang#82364 (Improve error msgs when found type is deref of expected) - rust-lang#82514 (Update Clippy) Failed merges: r? `@ghost` `@rustbot` modify labels: rollup

# Stabilization report ## Summary This stabilizes using macro expansion in key-value attributes, like so: ```rust #[doc = include_str!("my_doc.md")] struct S; #[path = concat!(env!("OUT_DIR"), "/generated.rs")] mod m; ``` See the changes to the reference for details on what macros are allowed; see Petrochenkov's excellent blog post [on internals](https://internals.rust-lang.org/t/macro-expansion-points-in-attributes/11455) for alternatives that were considered and rejected ("why accept no more and no less?") This has been available on nightly since 1.50 with no major issues. ## Notes ### Accepted syntax The parser accepts arbitrary Rust expressions in this position, but any expression other than a macro invocation will ultimately lead to an error because it is not expected by the built-in expression forms (e.g., `#[doc]`). Note that decorators and the like may be able to observe other expression forms. ### Expansion ordering Expansion of macro expressions in "inert" attributes occurs after decorators have executed, analogously to macro expressions appearing in the function body or other parts of decorator input. There is currently no way for decorators to accept macros in key-value position if macro expansion must be performed before the decorator executes (if the macro can simply be copied into the output for later expansion, that can work). ## Test cases - https://github.com/rust-lang/rust/blob/master/src/test/ui/attributes/key-value-expansion-on-mac.rs - https://github.com/rust-lang/rust/blob/master/src/test/rustdoc/external-doc.rs The feature has also been dogfooded extensively in the compiler and standard library: - rust-lang#83329 - rust-lang#83230 - rust-lang#82641 - rust-lang#80534 ## Implementation history - Initial proposal: rust-lang#55414 (comment) - Experiment to see how much code it would break: rust-lang#67121 - Preliminary work to restrict expansion that would conflict with this feature: rust-lang#77271 - Initial implementation: rust-lang#78837 - Fix for an ICE: rust-lang#80563 ## Unresolved Questions ~~rust-lang#83366 (comment) listed some concerns, but they have been resolved as of this final report.~~ ## Additional Information There are two workarounds that have a similar effect for `#[doc]` attributes on nightly. One is to emulate this behavior by using a limited version of this feature that was stabilized for historical reasons: ```rust macro_rules! forward_inner_docs { ($e:expr => $i:item) => { #[doc = $e] $i }; } forward_inner_docs!(include_str!("lib.rs") => struct S {}); ``` This also works for other attributes (like `#[path = concat!(...)]`). The other is to use `doc(include)`: ```rust #![feature(external_doc)] #[doc(include = "lib.rs")] struct S {} ``` The first works, but is non-trivial for people to discover, and difficult to read and maintain. The second is a strange special-case for a particular use of the macro. This generalizes it to work for any use case, not just including files. I plan to remove `doc(include)` when this is stabilized. The `forward_inner_docs` workaround will still compile without warnings, but I expect it to be used less once it's no longer necessary.

…=petrochenkov Stabilize extended_key_value_attributes Closes rust-lang#44732. Closes rust-lang#78835. Closes rust-lang#82768 (by making it irrelevant). # Stabilization report ## Summary This stabilizes using macro expansion in key-value attributes, like so: ```rust #[doc = include_str!("my_doc.md")] struct S; #[path = concat!(env!("OUT_DIR"), "/generated.rs")] mod m; ``` See Petrochenkov's excellent blog post [on internals](https://internals.rust-lang.org/t/macro-expansion-points-in-attributes/11455) for alternatives that were considered and rejected ("why accept no more and no less?") This has been available on nightly since 1.50 with no major issues. ## Notes ### Accepted syntax The parser accepts arbitrary Rust expressions in this position, but any expression other than a macro invocation will ultimately lead to an error because it is not expected by the built-in expression forms (e.g., `#[doc]`). Note that decorators and the like may be able to observe other expression forms. ### Expansion ordering Expansion of macro expressions in "inert" attributes occurs after decorators have executed, analogously to macro expressions appearing in the function body or other parts of decorator input. There is currently no way for decorators to accept macros in key-value position if macro expansion must be performed before the decorator executes (if the macro can simply be copied into the output for later expansion, that can work). ## Test cases - https://github.com/rust-lang/rust/blob/master/src/test/ui/attributes/key-value-expansion-on-mac.rs - https://github.com/rust-lang/rust/blob/master/src/test/rustdoc/external-doc.rs The feature has also been dogfooded extensively in the compiler and standard library: - rust-lang#83329 - rust-lang#83230 - rust-lang#82641 - rust-lang#80534 ## Implementation history - Initial proposal: rust-lang#55414 (comment) - Experiment to see how much code it would break: rust-lang#67121 - Preliminary work to restrict expansion that would conflict with this feature: rust-lang#77271 - Initial implementation: rust-lang#78837 - Fix for an ICE: rust-lang#80563 ## Unresolved Questions ~~rust-lang#83366 (comment) listed some concerns, but they have been resolved as of this final report.~~ ## Additional Information There are two workarounds that have a similar effect for `#[doc]` attributes on nightly. One is to emulate this behavior by using a limited version of this feature that was stabilized for historical reasons: ```rust macro_rules! forward_inner_docs { ($e:expr => $i:item) => { #[doc = $e] $i }; } forward_inner_docs!(include_str!("lib.rs") => struct S {}); ``` This also works for other attributes (like `#[path = concat!(...)]`). The other is to use `doc(include)`: ```rust #![feature(external_doc)] #[doc(include = "lib.rs")] struct S {} ``` The first works, but is non-trivial for people to discover, and difficult to read and maintain. The second is a strange special-case for a particular use of the macro. This generalizes it to work for any use case, not just including files. I plan to remove `doc(include)` when this is stabilized (rust-lang#82539). The `forward_inner_docs` workaround will still compile without warnings, but I expect it to be used less once it's no longer necessary.

rust-highfive assigned jyn514 Dec 30, 2020

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Dec 30, 2020

This comment has been minimized.

Sign in to view

jyn514 added C-cleanup Category: PRs that clean code up or issues documenting cleanup. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Dec 30, 2020

LeSeulArtichaut added the F-extended_key_value_attributes `#![feature(extended_key_value_attributes)] label Dec 30, 2020

LeSeulArtichaut mentioned this pull request Dec 31, 2020

ICE using #![feature(extended_key_value_attributes)] in an attribute for a macro #80540

Closed

LeSeulArtichaut added S-blocked Status: Marked as blocked ❌ on something else such as an RFC or other implementation work. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Dec 31, 2020

LeSeulArtichaut force-pushed the doc-include branch from 7291f9a to e7fb79c Compare January 10, 2021 14:16

LeSeulArtichaut added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-blocked Status: Marked as blocked ❌ on something else such as an RFC or other implementation work. labels Jan 10, 2021

This comment has been minimized.

Sign in to view

LeSeulArtichaut added S-blocked Status: Marked as blocked ❌ on something else such as an RFC or other implementation work. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 16, 2021

This comment has been minimized.

Sign in to view

LeSeulArtichaut force-pushed the doc-include branch from e7fb79c to 09780b3 Compare February 6, 2021 20:14

This comment has been minimized.

Sign in to view

Dylan-DPC-zz mentioned this pull request Feb 24, 2021

Rollup of 14 pull requests #82458

Closed

Dylan-DPC-zz pushed a commit to Dylan-DPC-zz/rust that referenced this pull request Feb 24, 2021

Rollup merge of rust-lang#80534 - LeSeulArtichaut:doc-include, r=jyn514

408ccd7

Use #[doc = include_str!()] in std cc rust-lang#78835 (comment) r? ``@jyn514``

Dylan-DPC-zz mentioned this pull request Feb 24, 2021

Rollup of 14 pull requests #82460

Closed

JohnTitor added a commit to JohnTitor/rust that referenced this pull request Feb 24, 2021

Rollup merge of rust-lang#80534 - LeSeulArtichaut:doc-include, r=jyn514

595b8be

Use #[doc = include_str!()] in std cc rust-lang#78835 (comment) r? ```@jyn514```

JohnTitor mentioned this pull request Feb 24, 2021

Rollup of 12 pull requests #82472

Closed

Dylan-DPC-zz pushed a commit to Dylan-DPC-zz/rust that referenced this pull request Feb 24, 2021

Rollup merge of rust-lang#80534 - LeSeulArtichaut:doc-include, r=jyn514

b3bcf48

Use #[doc = include_str!()] in std cc rust-lang#78835 (comment) r? ````@jyn514````

Dylan-DPC-zz mentioned this pull request Feb 24, 2021

Rollup of 13 pull requests #82481

Closed

Dylan-DPC-zz mentioned this pull request Feb 25, 2021

Rollup of 16 pull requests #82517

Merged

bors merged commit cb2b4ff into rust-lang:master Feb 25, 2021

rustbot added this to the 1.52.0 milestone Feb 25, 2021

jyn514 mentioned this pull request Mar 22, 2021

Stabilize extended_key_value_attributes #83366

Merged

LeSeulArtichaut deleted the doc-include branch August 25, 2021 14:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use #[doc = include_str!()] in std #80534

Use #[doc = include_str!()] in std #80534

LeSeulArtichaut commented Dec 30, 2020

This comment has been minimized.

LeSeulArtichaut commented Dec 30, 2020 •

edited

Loading

petrochenkov commented Dec 30, 2020

LeSeulArtichaut commented Dec 30, 2020 •

edited

Loading

jyn514 commented Dec 30, 2020

LeSeulArtichaut commented Dec 30, 2020

LeSeulArtichaut commented Dec 30, 2020

LeSeulArtichaut commented Dec 31, 2020 •

edited

Loading

LeSeulArtichaut commented Jan 10, 2021

This comment has been minimized.

LeSeulArtichaut commented Jan 10, 2021

jyn514 commented Jan 10, 2021

LeSeulArtichaut commented Jan 16, 2021

This comment has been minimized.

This comment has been minimized.

jyn514 commented Feb 6, 2021

Use #[doc = include_str!()] in std #80534

Use #[doc = include_str!()] in std #80534

Conversation

LeSeulArtichaut commented Dec 30, 2020

This comment has been minimized.

LeSeulArtichaut commented Dec 30, 2020 • edited Loading

petrochenkov commented Dec 30, 2020

LeSeulArtichaut commented Dec 30, 2020 • edited Loading

jyn514 commented Dec 30, 2020

LeSeulArtichaut commented Dec 30, 2020

LeSeulArtichaut commented Dec 30, 2020

LeSeulArtichaut commented Dec 31, 2020 • edited Loading

LeSeulArtichaut commented Jan 10, 2021

This comment has been minimized.

LeSeulArtichaut commented Jan 10, 2021

jyn514 commented Jan 10, 2021

LeSeulArtichaut commented Jan 16, 2021

This comment has been minimized.

This comment has been minimized.

jyn514 commented Feb 6, 2021

LeSeulArtichaut commented Dec 30, 2020 •

edited

Loading

LeSeulArtichaut commented Dec 30, 2020 •

edited

Loading

LeSeulArtichaut commented Dec 31, 2020 •

edited

Loading