Skip to content

feat: register C/C++ and Rust macro definitions as Function nodes with resolvable invocations#672

Merged
vitali87 merged 3 commits into
mainfrom
feat/macros-as-function-nodes
Jul 9, 2026
Merged

feat: register C/C++ and Rust macro definitions as Function nodes with resolvable invocations#672
vitali87 merged 3 commits into
mainfrom
feat/macros-as-function-nodes

Conversation

@vitali87

@vitali87 vitali87 commented Jul 9, 2026

Copy link
Copy Markdown
Owner

Summary

Macros are modeled cross-language as Function nodes (the schema decision: no language-specific label). Previously macro definitions were invisible in every language: Rust macro_invocation sites were captured but had no definition node to bind to, and the C++ libclang frontend never saw preprocessor entities at all. Now C/C++ #define and Rust macro_rules! register as Functions, invocations resolve to them, and dead-code treats macros like any function.

Rust

  1. Definitions (test_rs_macro_definition_nodes.py): macro_definition joins SPEC_RS_FUNCTION_TYPES and the function query, so macro_rules! square registers crate.module.square as a Function.
  2. Invocations: square!(3) was captured as a call but dropped nameless -- macro_invocation carries its callee in the macro field, which _get_call_target_name never read. A dedicated case extracts it, so the invocation resolves through the normal chain.
  3. Namespace separation (test_macro_and_fn_namespaces_do_not_cross_bind): Rust macros and functions live in separate namespaces. Without a gate, std-prelude write!(f, ..) would bind a same-module fn write (a false edge that revives dead code), and trace(x) would bind a same-module macro_rules! trace. Definition ingest records macro qns (DefinitionProcessor.macro_qns); Pass 3 requires macro-invocation call sites to bind macro targets and fn-namespace call sites to avoid them.
  4. #[macro_export] (test_macro_export_attribute_marks_exported): macro_rules! takes no pub; the attribute is what publishes a macro (to the crate root) as library API, so it now sets is_exported -- an exported but internally-uninvoked macro must not report dead.

C/C++ (libclang frontend)

  1. Definitions (test_cpp_frontend_macros.py): TUs parse with PARSE_DETAILED_PROCESSING_RECORD; repo MACRO_DEFINITION cursors become Function nodes with Module DEFINES edges. Compiler builtins (no file), system-header macros (outside the repo), and empty-bodied object-like macros (include guards, feature flags -- extent covers only the name) are skipped.
  2. Invocations: MACRO_INSTANTIATION.referenced gives the exact definition (libclang resolved it). Macro cursors are TU-level preprocessing entities, never AST-nested, so the caller is recovered at flush by tightest enclosing function/method span in the use-site file; a use outside any span attributes to the Module, mirroring the existing module-caller rule.

Verification (RED -> GREEN)

  • 5 new tests across 2 files, each confirmed failing before its fix.
  • Full suite: 4689 passed, 0 failed.
  • Dead-code evals: mini-redis 0 -> 0; anyhow (defines anyhow!/bail!/ensure!/cfg-duplicated backtrace! and invokes them internally) reports a byte-identical dead set vs main -- all newly-registered macros are live via roots or resolved invocations, zero regressions.
  • ruff clean; ty diagnostic count identical to main.

@vitali87

vitali87 commented Jul 9, 2026

Copy link
Copy Markdown
Owner Author

@greptile review

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds support for C/C++ and Rust macro definitions and instantiations, registering them as Function nodes and resolving their call sites appropriately. It also handles namespace separation in Rust to prevent false bindings between functions and macros, and adds comprehensive tests. The review feedback highlights a few robust improvements: skipping tree-sitter comment nodes when detecting Rust macro exports, guarding against potential null spelling values in Clang cursors, and using safe dictionary lookups for node properties to avoid KeyErrors.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread codebase_rag/parsers/export_detection.py Outdated
Comment thread codebase_rag/parsers/cpp_frontend/frontend.py
Comment thread codebase_rag/parsers/cpp_frontend/frontend.py Outdated
@greptile-apps

greptile-apps Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds macro definitions and invocations to the graph model. The main changes are:

  • Rust macro_rules! definitions are registered as Function nodes with is_macro.
  • Rust macro invocations now resolve through the normal call path while staying separate from function-name resolution.
  • #[macro_export] marks Rust macro definitions as exported roots.
  • C/C++ libclang parsing now includes preprocessing records for repository macro definitions and instantiations.
  • C/C++ macro uses emit CALLS edges from the enclosing function or module.
  • Incremental rehydration restores is_macro state for unchanged files.
  • Tests, schema metadata, documentation, and eval support were updated for macro handling.

Confidence Score: 5/5

Safe to merge with minimal risk.

No blocking issues were found in the changed paths. The PR includes focused coverage for Rust macro definitions, invocation resolution, namespace separation, exported macros, incremental rehydration, and C++ frontend macro calls.

No files require special attention.

T-Rex T-Rex Logs

What T-Rex did

  • T-Rex documented the environment blockers encountered when preparing the Python virtual environment, including a broken prior venv, missing pytest in a fresh uv venv, missing loguru under system Python, and missing pydantic_ai via the conftest patch target.
  • T-Rex captured the environment setup step that created the virtual environment and installed the minimal uv package into the .venv.
  • T-Rex ran the final test suite with PYTHONPATH set and the two macro-related test modules, and recorded the results showing 6 passed and 1 warning with exit code 0.

View all artifacts

T-Rex Ran code and verified through T-Rex

Important Files Changed

Filename Overview
codebase_rag/parsers/cpp_frontend/frontend.py Enables libclang preprocessing records, registers repository macros as Function nodes, and emits macro CALLS edges from enclosing scopes.
codebase_rag/parsers/call_processor.py Extracts Rust macro invocation names and gates Rust resolution across macro/function namespaces.
codebase_rag/parsers/function_ingest.py Persists Rust macro Function nodes with is_macro and records their namespace membership.
codebase_rag/graph_updater.py Rehydrates persisted macro markers into the definition processor during incremental indexing.
codebase_rag/parsers/export_detection.py Treats Rust #[macro_export] macro definitions as exported API roots.
codebase_rag/tests/test_rs_macro_definition_nodes.py Adds Rust coverage for macro definition nodes, invocation resolution, namespace separation, incremental rehydration, and #[macro_export].
codebase_rag/tests/test_cpp_frontend_macros.py Adds libclang frontend coverage for C++ macro definitions and macro invocation CALLS edges.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
participant Parser as Rust/libclang parser
participant Def as Definition ingest
participant Reg as Function registry + macro_qns
participant Calls as Call processor/frontend flush
participant Graph as Graph ingestor

Parser->>Def: Capture macro definition
Def->>Reg: Register Function qn with is_macro
Def->>Graph: MERGE Function and DEFINES edge
Parser->>Calls: Capture macro invocation
Calls->>Reg: Resolve target and validate macro namespace
Calls->>Graph: Emit CALLS edge to macro Function
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
participant Parser as Rust/libclang parser
participant Def as Definition ingest
participant Reg as Function registry + macro_qns
participant Calls as Call processor/frontend flush
participant Graph as Graph ingestor

Parser->>Def: Capture macro definition
Def->>Reg: Register Function qn with is_macro
Def->>Graph: MERGE Function and DEFINES edge
Parser->>Calls: Capture macro invocation
Calls->>Reg: Resolve target and validate macro namespace
Calls->>Graph: Emit CALLS edge to macro Function
Loading

Reviews (3): Last reviewed commit: "fix: rehydrate macro namespace on increm..." | Re-trigger Greptile

@greptile-apps

greptile-apps Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR registers C/C++ and Rust macro definitions as Function nodes and resolves macro invocations. The main changes are:

  • Adds Rust macro_definition capture and macro_invocation target extraction.
  • Separates Rust macro and function namespaces during call resolution.
  • Marks Rust #[macro_export] macros as exported API roots.
  • Enables libclang detailed preprocessing records for C/C++ macro definitions and instantiations.
  • Adds regression tests for Rust macro nodes, namespace binding, macro exports, and C++ frontend macros.

Confidence Score: 4/5

Mostly safe, with one contained indexing bug to fix before relying on incremental Rust macro results.

Clean-index macro handling is covered by focused tests. Incremental indexing can drop calls to unchanged Rust macros because macro namespace metadata is not rehydrated.

codebase_rag/parsers/call_processor.py; codebase_rag/graph_updater.py rehydration metadata for macro definitions

T-Rex T-Rex Logs

What T-Rex did

  • The attempt to rehydrate the macro namespace was blocked after reaching the maximum allowed execution steps, so a runtime reproduction harness or artifacts could not be created.
  • Direct command and environment evidence confirm the exact requested commands and setup, a test-only CLI stub harness was added to satisfy patching, and the wrapper run completed with 5 tests passing.

View all artifacts

T-Rex Ran code and verified through T-Rex

Important Files Changed

Filename Overview
codebase_rag/parsers/call_processor.py Extracts Rust macro invocation names and gates Rust macro/function namespace binding; incremental rehydration can drop calls to unchanged macros.
codebase_rag/parsers/cpp_frontend/frontend.py Enables detailed preprocessing records, emits macro Function nodes, and resolves macro instantiation callers by span.
codebase_rag/parsers/definition_processor.py Tracks Rust macro Function qualified names for namespace-aware call resolution; metadata is not rehydrated on incremental runs.
codebase_rag/parsers/function_ingest.py Registers macro_definition qns in the macro namespace set during Rust function ingest.
codebase_rag/tests/test_cpp_frontend_macros.py Adds C++ frontend macro definition and invocation regression tests.
codebase_rag/tests/test_rs_macro_definition_nodes.py Adds Rust macro definition, invocation namespace, and macro_export tests.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
participant Def as Definition Pass
participant Reg as Function Registry
participant MacroSet as macro_qns
participant Calls as Call Pass
participant Graph as Graph Ingestor

Def->>Reg: "register macro_rules! / #define as Function"
Def->>MacroSet: record Rust macro qns
Calls->>Calls: extract macro invocation target
Calls->>Reg: resolve call name
Calls->>MacroSet: validate Rust macro/function namespace
Calls->>Graph: emit CALLS edge to macro Function
Note over Graph: C/C++ frontend emits macro definitions and instantiations directly
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
participant Def as Definition Pass
participant Reg as Function Registry
participant MacroSet as macro_qns
participant Calls as Call Pass
participant Graph as Graph Ingestor

Def->>Reg: "register macro_rules! / #define as Function"
Def->>MacroSet: record Rust macro qns
Calls->>Calls: extract macro invocation target
Calls->>Reg: resolve call name
Calls->>MacroSet: validate Rust macro/function namespace
Calls->>Graph: emit CALLS edge to macro Function
Note over Graph: C/C++ frontend emits macro definitions and instantiations directly
Loading
Prompt To Fix All With AI
Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
codebase_rag/parsers/call_processor.py:1758-1760
**Rehydrate macro namespace**
Incremental runs rehydrate unchanged Function nodes into `function_registry` but never repopulate `DefinitionProcessor.macro_qns`. When a changed Rust file calls a `macro_rules!` definition from an unchanged file, resolution returns that macro qn, this check treats it as a non-macro target, and drops the `CALLS` edge. Clean and incremental indexes then disagree, and the unchanged macro can be reported dead.

### Agentic Framework
-... ([source](https://app.greptile.com/graph-code/github/vitali87/code-graph-rag/-/custom-context?memory=d4240b05-b763-467a-a6bf-94f73e8b6859))

Reviews (2): Last reviewed commit: "docs: document macros-as-Function schema..." | Re-trigger Greptile

Comment thread codebase_rag/parsers/call_processor.py
@codecov-commenter

codecov-commenter commented Jul 9, 2026

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 95.52239% with 9 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
codebase_rag/parsers/cpp_frontend/frontend.py 86.36% 9 Missing ⚠️

📢 Thoughts on this report? Let us know!

@vitali87

vitali87 commented Jul 9, 2026

Copy link
Copy Markdown
Owner Author

@greptile review

@sonarqubecloud

sonarqubecloud Bot commented Jul 9, 2026

Copy link
Copy Markdown

@vitali87
vitali87 merged commit 92559c6 into main Jul 9, 2026
22 checks passed
@vitali87
vitali87 deleted the feat/macros-as-function-nodes branch July 9, 2026 10:27
@vitali87

vitali87 commented Jul 9, 2026

Copy link
Copy Markdown
Owner Author

Re the Codecov report (9 missing lines in cpp_frontend/frontend.py): addressed in #673. Seven lines are now covered by new fixtures (Module-caller fallback for file-scope macro uses, out-of-repo and ignored-dir use sites, command-line and system macros) plus a guard reorder that removed an unreachable check. The two remaining lines are defensive guards against libclang returning None cursors (MACRO_INSTANTIATION with no file / no referenced definition); they are not constructible from source-level fixtures and are kept so unusual TUs skip instead of crash. The SonarQube S3776 issue on _register_function is also fixed there.

vitali87 added a commit that referenced this pull request Jul 9, 2026
test(cpp): address codecov and sonar reports from #672 (macro guard coverage, complexity)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants