Fix ICE when rustdoc is scraping examples inside of a proc macro

[willcrichton]

This PR provides a clearer semantics for how --scrape-examples interacts with macros. If an expression's span AND it's enclosing item's span both are not from_expansion, then the example will be scraped. The added test case rustdoc-scrape-examples-macros shows a variety of situations.

• A macro-rules macro that takes a function call as input: good
• A macro-rules macro that generates a function call as output: bad
• A proc-macro that generates a function call as output: bad
• An attribute macro that generates a function call as output: bad
• An attribute macro that takes a function call as input: good, if the proc macro is designed to propagate the input spans

I ran this updated rustdoc on pyo3 and confirmed that it successfully scrapes examples from inside a proc macro, eg

(cc @mejrs)

Additionally, this PR fixes an ordering bug in the highlighting logic.

Fixes #90567.

r? @jyn514

[jyn514]

An attribute macro that takes a function call as input: good, if the proc macro is designed to propagate the input spans

Can you also test what happens if it's buggy and doesn't propagate spans? It seems fine to discard that example, I just want to make sure it doesn't cause an ICE.

[camelid]

Additionally, this PR fixes an ordering bug in the highlighting logic.

Could you separate that into another commit? It'll make it easier to understand why code was changed later on.

And thanks for adding a test ❤️

[jyn514]

For posterity, this fixes the following ICE when running on Pyo3 (reported on discord):

$ cargo +nightly doc -Z unstable-options -Z rustdoc-scrape-examples=examples
   Compiling pyo3 v0.14.5 (C:\Users\user\rust\pyo3)
thread 'rustc' panicked at 'assertion failed: enclosing_item_span.contains(expr_span)', src\librustdoc\scrape_examples.rs:93:9
stack backtrace:
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

error: internal compiler error: unexpected panic

error: Unrecognized option: 'scrape-examples-output-path'

error: could not document `pyo3

[camelid]

(FYI, I just created an A-rustdoc-scrape-examples label to use in tracking this feature. I'll add it to some other PRs and issues too.)

[willcrichton]

Could you separate that into another commit? It'll make it easier to understand why code was changed later on.

@camelid I've separated this into two commits.

Can you also test what happens if it's buggy and doesn't propagate spans? It seems fine to discard that example, I just want to make sure it doesn't cause an ICE.

@jyn514 the rustdoc-scrape-examples-macro test already does this, if I'm understanding you correctly. The #[an_attr_macro] macro is used twice, the second time with #[an_attr_macro(with_span)].

[jyn514]

r=me with CI passing :)

@bors delegate=willcrichton

[bors]

✌️ @willcrichton can now approve this pull request

[willcrichton]

@bors r=jyn514

[bors]

📌 Commit 4846d10 has been approved by jyn514

[mejrs]

Is this supposed to draw code examples from /tests? That doesn't sound right.

[bors]

⌛ Testing commit c62817b with merge 86e2b3bd0cb2eff1a3502fd66296a3f3d8e480da...

[bors]

💔 Test failed - checks-actions

[jyn514]

@willcrichton every platform names the suffix for dynamic libraries something different - you can either try to make it work in the makefile (with $(RUSTC) --crate-name foo --crate-type dylib --print file-names, see the trouble I've been having in https://github.com/rust-lang/rust/pull/88368/files#diff-33be197db3e3ddb089ba3380ea3d6ce10944ff5517531547039f0fa6c06216e3) or just make it linux-only.

[willcrichton]

Ok I did your suggested change @jyn514.

[jyn514]

@bors r+

Thanks!

[bors]

📌 Commit 82b23be has been approved by jyn514

[bors]

⌛ Testing commit 82b23be with merge 0d1754e...

[bors]

☀️ Test successful - checks-actions
Approved by: jyn514
Pushing 0d1754e to master...

[rust-timer]

Finished benchmarking commit (0d1754e): comparison url.

Summary: This change led to moderate relevant improvements 🎉 in compiler performance.

• Moderate improvement in instruction counts (up to -0.7% on full builds of webrender)

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

@rustbot label: -perf-regression

[camelid]

I guess fetching the span was quite slow and moving it to after the expansion check (and other checks) improved performance?