Python AST extraction creates duplicate 'ghost' nodes for cross-file class references (dataclasses, type annotations)

[SatishRockzz]

Summary

Running the graphify skill's Python AST extraction (graphify.extract.extract_python) on a corpus where a dataclass/class is defined in one file and referenced (imported + used in a type annotation, e.g. a function return type or another dataclass's field type) from a different file produces two separate nodes with the same label — one at the defining file's correct node ID, and one "ghost" duplicate anchored at the referencing file, instead of a single node with an imports/references edge pointing at it.

This inflates degree/centrality metrics (a class can show up as a "god node" partly due to its own duplicate's edges being counted separately) and fragments community detection (each ghost duplicate can end up isolated in its own thin community).

Reproduction

A minimal repro, mirroring what I hit:

# pkg/a/base.py
from dataclasses import dataclass

@dataclass
class PricePoint:
    value: float

# pkg/b/consumer.py
from pkg.a.base import PricePoint

def latest(rows: list[PricePoint]) -> PricePoint | None:
    ...

Run graphify's extraction over both files. Expected: one node for PricePoint (anchored at pkg/a/base.py), with an imports/uses edge from consumer.py's file node (and/or from latest()) to it.

Observed (real corpus, not the minimal repro above — I didn't isolate a standalone reproduction script, but the pattern below is exactly this shape): I got 5 separate nodes all labeled PricePoint across a 5-file corpus where one file defines it and 4 others import + reference it in type annotations:

agri_baseline_pricepoint                                  <- correct, at the defining file
signal_intelligence_connections_prices_py_pricepoint      <- ghost duplicate
signal_intelligence_fronts_agri_deviation_py_pricepoint   <- ghost duplicate
signal_intelligence_fronts_upstream_watch_signals_py_pricepoint <- ghost duplicate
signal_intelligence_signal_resolution_py_pricepoint       <- ghost duplicate

The 4 ghost duplicates all share one wrong-ID pattern: signal_intelligence_<full_relative_path_with_underscores>_py_pricepoint — i.e. the full path from the project root gets baked into the ID, while the correct/defining node instead uses the short {immediate_parent_dir}_{filename_stem}_{entity} scheme (agri_baseline_pricepoint). These are two different ID-generation schemes colliding.

I found 7 more instances of the same pattern in the same run, but with the other malformed variant — a bare, unprefixed lowercase name instead of a fully-qualified one:

BaselineResult -> agri_baseline_baselineresult (correct)  vs  baselineresult (ghost, from a different file's type annotation)
GapClass       -> ledger_quality_gapclass (correct)       vs  gapclass (ghost, referenced from a file in a different top-level subdirectory)
DayRecord      -> agri_ingest_dayrecord (correct)          vs  dayrecord (ghost)
DeviationRecord-> agri_deviation_deviationrecord (correct) vs  deviationrecord (ghost)
GateResult     -> ledger_risk_gateresult (correct)         vs  gateresult (ghost)
ChokepointDay  -> upstream_watch_ingest_chokepointday (correct) vs  chokepointday (ghost)
PotashMonth    -> upstream_watch_ingest_potashmonth (correct)   vs  potashmonth (ghost)

All 11 ghost duplicates were verified against source: every one of these classes is defined exactly once (confirmed via grep -n "^class "), and the "extra" nodes are 100% attributable to a different file's import + type-annotation usage, never a genuine second definition.

What I ruled out

I confirmed this is not a semantic-subagent/LLM extraction issue: I re-ran the semantic (Claude subagent) extraction pass with an explicit canonical-ID glossary telling subagents exactly which ID to use for each of these 24 classes, and the corrected subagent output had zero duplicate nodes for any of them. I then diffed the subagents' new output against .graphify_ast.json (the structural, non-LLM extraction) and found all 11 duplicate IDs originate purely from the AST pass, confirming the bug is in extract_python's (or the shared cross-file resolution pass around resolve_exported_origin/facts.uses in extract.py) handling of Python — not the LLM layer.

Suspected area

graphify/extract.py:

• _python_collect_type_refs (~line 477) walks a Python type annotation and appends (name, "type") — just the raw identifier text, not yet resolved to a file.
• The cross-file resolution path around resolve_exported_origin / facts.uses / local_aliases_by_file (~line 6874 onward) looks primarily written for JS/TS ES-module semantics (named exports, star exports) and may not correctly resolve a Python from X import Y alias back to Y's defining-file node ID in all cases — falling back to a synthesized ID (either the referencing file's own full-path-qualified stem, or no prefix at all) instead of failing over to a "no edge" or a proper lookup.

I haven't isolated the exact branch producing each of the two different malformed-ID shapes (full-path vs bare), so treat the above as a strong lead rather than a confirmed line number.

Suggested fix

• When a Python type annotation reference (_python_collect_type_refs output) can't be resolved to an existing symbol node via the file's own from X import Y aliases, either (a) resolve it properly against the corpus-wide symbol table keyed by (defining_file, class_name), or (b) don't synthesize a new node ID at all — drop the edge rather than create a duplicate node with a fabricated ID.
• Add a post-extraction consistency check: if two nodes share an identical label and one of them has zero real definition-site metadata (no contains edge from a file node, no docstring, etc.), that's a strong signal one of them is a ghost duplicate worth auto-merging or at least flagging in GRAPH_REPORT.md's "Knowledge Gaps" section.

Environment

• graphifyy==0.8.28 (via uv tool install graphifyy)
• macOS (Darwin 25.6.0, arm64), Python 3.14 (the tool's own venv)
• Corpus: 20 Python files + 1 markdown doc, real project code (not synthetic), all files under one repo root

Happy to share the full graphify-out/graph.json (before my manual post-processing merge) if useful — didn't attach it here since it's from a private-ish personal project, but I can redact/trim it to just the relevant nodes/edges if that helps debug.

[Rishet11]

Looking into this — here's what I measured on current v8 (3b243d6, 0.9.31) versus the 0.8.28 you reported against.

The duplication itself no longer reproduces for the unambiguous shapes. I built your exact corpus layout (agri/baseline.py defining PricePoint, signal_intelligence/connections/prices.py and signal_intelligence/fronts/agri_deviation.py importing it and using it in parameter/return annotations) and ran extract():

1 PricePoint node: agri_baseline_pricepoint  (source_file=baseline.py)
  signal_intelligence_connections_prices_g   -> agri_baseline_pricepoint  references parameter_type
  signal_intelligence_connections_prices_g   -> agri_baseline_pricepoint  references return_type
  signal_intelligence_fronts_agri_deviation_h -> agri_baseline_pricepoint references parameter_type
  signal_intelligence_fronts_agri_deviation_h -> agri_baseline_pricepoint references return_type

No ghosts, in either the bare or the full-path shape. Same result with 1 referencing file, 4 referencing files, a class Sub(PricePoint) superclass reference, a method-level parameter annotation, and even an annotation with no import statement at all. ensure_named_node (graphify/extractors/engine.py:2393) now mints a sourceless stub, and _rewire_unique_stub_nodes (graphify/extract.py:1893) collapses it onto the real definition. That path was hardened after 0.8.28.

But two real defects survive, and they are both in your title.

1. Dataclass / class-body field annotations emit no edge at all.

# pkg/b/q.py
from pkg.a.base import PricePoint

@dataclass
class Quote:
    point: PricePoint

Result: zero references edges. Not a ghost node — a silently dropped edge. _python_collect_type_refs (engine.py:82) has exactly two call sites, function parameters (engine.py:3350) and return types (engine.py:3360). Nothing walks a Python class_definition body for annotated assignments. Java and TS both handle their equivalent, and I confirmed they emit the edge cross-file today:

JAVA  b_quote_quote -> a_pricepoint_pricepoint  references field
TS    b_quote_quote -> a_base_pricepoint        references field
PY    (nothing)

2. When the simple name is genuinely ambiguous, the bare ghost still appears — and this matches your 7 bare-lowercase cases.

Two real PricePoint definitions in different packages, one file importing a specific one:

3 PricePoint nodes:
  pkg_a_base_pricepoint   src=base.py
  pkg_z_other_pricepoint  src=other.py
  pricepoint              src=''        <- ghost
  pkg_b_c_f -> pricepoint  references parameter_type

_rewire_unique_stub_nodes bails when len(candidates) != 1, so the stub survives and the edge stays on it, even though from pkg.a.base import PricePoint names the defining module unambiguously. Java does not have this problem, because _resolve_java_type_references (graphify/extractors/resolution.py:2255) re-points the edge using the file's own import statements. Same for PHP (:2417) and C#. Python has no equivalent resolver — _collect_python_symbol_resolution_facts (:1791) collects only imports and calls, never type references. Java on the identical corpus:

2 PricePoint nodes, no ghost
  b_svc_svc_take -> a_pricepoint_pricepoint  references parameter_type   <- correct one, via the import

So this is the familiar "language X already does this, Python was never wired up" shape, twice over.

I'd like to take this. Plan is two commits: the class-body field-annotation walk (mirroring the Java record-component branch at engine.py:2845 and TS's field branch at resolution.py:1450), and a Python import-aware type resolver mirroring _resolve_java_type_references. Tests in a new tests/test_python_type_references.py, plus a real-CLI test asserting against the written graph.json, and a regression test pinning the now-fixed unambiguous case so it cannot silently come back.

One note for bookkeeping: #1231 targets this same area and has been conflicting since June. I'm not building on it; happy to close mine if you'd rather revive that one.

Could you share the redacted graph.json for the 4 path-shaped signal_intelligence_*_pricepoint ghosts? If those classes are each defined exactly once, that shape should be gone at 0.9.31, and I'd like to know what in your corpus still triggers it that my repro doesn't.