Skip to content

feat(preproc): build macro expansion provenance model#237

Merged
hongjr03 merged 68 commits into
masterfrom
feat/preproc-provenance-alignment
Jun 8, 2026
Merged

feat(preproc): build macro expansion provenance model#237
hongjr03 merged 68 commits into
masterfrom
feat/preproc-provenance-alignment

Conversation

@hongjr03
Copy link
Copy Markdown
Member

@hongjr03 hongjr03 commented Jun 6, 2026

Summary

This PR builds a source-aware macro expansion provenance path from the preprocessor trace through HIR and into IDE features. The preprocessor model now tracks macro definitions, references, calls, recursive expansions, emitted tokens, token provenance, include chains, and inactive ranges as structured facts. Unsupported transformed-token provenance, unmapped sources, display-only expansion sources, and ambiguous multi-context queries are represented explicitly instead of being collapsed into empty results or text-search recovery.

HIR maps those facts into repository-facing source concepts. Real source files, file-backed virtual sources, display-only virtual expansion text, configured predefines, include buffers, and unmapped sources are represented distinctly. Expansion display text is exposed under /__vide/preproc/<profile>/expansion/<expansion-id>.sv for hover and range mapping, but is not treated as a parse-stable source buffer unless a file-backed virtual source exists.

IDE consumers now go through hir::preproc for macro/include facts. This keeps goto, references, hover, inactive-branch handling, and diagnostic target selection on the same authority-driven path. Macro hover now presents the user-facing expansion story: signature, argument bindings, final expanded result, and each nested expansion step, while avoiding internal virtual-buffer or per-token provenance details.

Data Flow

flowchart LR
    A["Source files, include buffers, and configured predefines"] --> B["Source-aware preprocessor trace"]
    B --> C["Preproc semantic model"]
    C --> C1["Macro definitions and references"]
    C --> C2["Macro calls and expansion graph"]
    C --> C3["Emitted tokens and token provenance"]
    C --> C4["Include chains and inactive ranges"]

    C1 --> D["HIR preproc source map"]
    C2 --> D
    C3 --> D
    C4 --> D

    D --> E["MappedPreprocSource"]
    E --> E1["RealFile"]
    E --> E2["VirtualFile file-backed virtual source"]
    E --> E3["VirtualDisplay display-only expansion text"]
    E --> E4["Unmapped source"]

    E3 --> F["/__vide/preproc/<profile>/expansion/<expansion-id>.sv"]
    D --> G["HIR node to expanded token range"]
    G --> H["Token provenance"]

    H --> I["source token"]
    H --> J["macro call"]
    H --> K["macro body"]
    H --> L["macro argument"]
    H --> M["predefine / builtin"]
    H --> N["unsupported / unavailable"]

    D --> O["hir::preproc facade"]
    O --> P["goto / references / hover"]
    O --> Q["diagnostic target selection"]
    O --> R["macro expansion hover"]
Loading

User-Facing Behavior

  • Macro-generated HIR nodes are not treated as ordinary real-file nodes.
  • Macro arguments can participate in normal definition/reference flows through expanded HIR.
  • Configured macros with manifest provenance can navigate to vide.toml when an authoritative manifest range exists.
  • Hover can explain macro expansion as signature, arguments, final result, and nested steps.
  • Diagnostics select a reasonable provenance target and return unavailable when the mapping is not precise enough.

Validation

  • cargo fmt --check
  • cargo test -p preproc
  • cargo test -p hir preproc -- --nocapture
  • cargo test -p ide preproc -- --nocapture
  • cargo check -p ide
  • cargo check -p vide
  • git diff --check

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jun 6, 2026

Docs preview: https://vide.pascal-lab.net/preview/pr-237/

hongjr03 added 26 commits June 8, 2026 01:53
@hongjr03 hongjr03 merged commit 33a3013 into master Jun 8, 2026
12 checks passed
@hongjr03 hongjr03 deleted the feat/preproc-provenance-alignment branch June 8, 2026 06:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant