Make -Zcache-proc-macros survive span shifts [touch -21%; shift edits: zero hits -> -10% vs uncached] - #93
Draft
xmakro wants to merge 1 commit into
Conversation
This was referenced Aug 4, 2026
Draft
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Split from #88 (integrated stack and cumulative numbers there). Based on the attr-arg span parenting branch, which provides the
TokenStream::map_spansutility; everything else is flag-gated and independent of the other siblings.-Zcache-proc-macroscurrently gets zero cache hits on any edit that shifts positions: the query key is(LocalExpnId, &TokenStream), and both components are position-fragile. Expansion indices shift whenever an edit changes how many expansions precede a derive (measured: a one-line comment insertion at the top of the file misses 598 of 600 cached expansions), and the token spans in the key shift with the edit. On a 300-struct serde crate the flag was a net regression on such edits: 42.0B instructions with the flag vs 41.3B without.Three changes, all in the flag path:
DepNodeKeyimpl fingerprints the key as (parent moduleDefPathHash, macroDefPathHash, tokens hashed with span hashing disabled). The triple is collision-free for legal code: identical tokens include the item name, and duplicate names in one module are rejected, so no two live expansions can produce the same fingerprint.LocalExpnIddeliberately does not enter the hash.ExpnId::stable_hashasserts default hashing controls, so the pieces are hashed manually rather than hashing the whole key underhash_spans = false.no_hashon the query. A span-only difference in the output tokens is deliberately treated as no change; that is the contract that makes reuse across shifts possible. Without it,incremental_verify_ich(which re-hashes roughly 1 in 32 cache loads even when the flag is off) panics with unstable fingerprints when the reused output's spans have shifted.source_spanof every generated def move with edits, which re-runs typeck and MIR building for all generated bodies; the file-start anchor keeps them position-stable.Perf (300-struct serde crate, instructions): touch rebuilds 8.25B to 6.50B with the flag (-21%; the serde_derive dylib run alone is 16% of rustc time). Span-shift edits go from zero hits (+1.9% vs no flag) to hitting: -10% vs no flag measured standalone on the original base; on top of the full #88 stack the same edit is 9.05B with the flag vs 10.9B without. Adds an incremental test that a span shift reuses the cached expansion (
loaded_from_disk).Tradeoff, flag-only: errors reported inside a cached expansion lose their exact position; help notes and expansion notes stay correct and line accurate because
ExpnDatais up to date. Flag-off diagnostics are unchanged.Validation (as part of the #88 stack): tests/incremental clean, full tests/ui clean (21309 tests), forced
-Zincremental-verify-ichacross successive span shifts with the flag on with no panics, diagnostic fidelity checked for an error inside a cached expansion, serde_json output of a cached-then-shifted build round-trips correctly, mid-file derive insertion correctly misses instead of reusing stale output.Depends on #89 (provides map_spans; this PR is based on that branch). Siblings: #90, #91, #92.