Full `graphify update` silently evicts LLM semantic edges from re-extracted Markdown/doc sources (edge eviction is provenance-blind)

[mphstudios]

Summary

A full graphify update (e.g. bare graphify update ., or the post-checkout hook) silently evicts LLM-extracted semantic edges (semantically_similar_to, conceptually_related_to, etc.) whose source_file is a document that also has an AST extractor — most notably Markdown (.md, .mdx, .qmd). The concept nodes survive but the semantic edges connecting them are dropped, leaving orphaned nodes.

This contradicts the stated intent in the code itself. _reconcile_existing_graph in watch.py comments that "edges from unchanged or semantic sources survive", and node preservation is deliberately provenance-aware ("Semantic nodes lack the AST origin marker and remain preserved"). Edge preservation is not provenance-aware, so on a full rebuild the intent is not met.

Environment

• graphify (graphifyy) 0.9.14, Python 3.13, macOS.

Root cause

In graphify/watch.py, _reconcile_existing_graph():

• Node eviction is provenance-aware. A node is only evicted if it is an AST-origin node (node.get("_origin") == "ast") owned by a re-extracted source. Semantic nodes (no _origin == "ast" marker) are preserved.
• Edge eviction is provenance-blind. An edge is evicted purely by source_file membership: is_evicted(edge, edge_evicted_source_identities), where on a full rebuild edge_evicted_source_identities includes the identity of every re-extracted source (rebuilt_source_identities = all extract_targets).

extract_targets on a full rebuild includes every code file and every document file that has an AST extractor — the loop that appends .md/.mdx/.qmd to code_files. So a semantic edge whose source_file is a Markdown doc is treated as "owned by a re-extracted source" and evicted, even though the AST pass that re-extracted the file does not (and cannot) regenerate that semantic edge — it was produced by the LLM pass, which graphify update does not run.

The AST pass legitimately owns and replaces a file's AST edges. It should not own that file's semantic edges. Node reconciliation already draws this distinction via _origin; edge reconciliation does not.

Reproducer

Two Markdown docs describing the same idea with no structural link between them, so a real extraction emits a semantically_similar_to edge:

auth.md

# Token Validation
The access layer verifies a bearer token before granting a request. It checks the
token signature against the signing key, confirms the token has not expired, and
rejects any token whose audience claim does not match this service.

login.md

# Session Verification
When a login session is presented, the gateway decides whether it is still
trustworthy. It recomputes the session's signature from the secret, ensures the
session's expiry timestamp is in the future, and denies a session issued for a
different application.

1. AST-extract both, run a real semantic extraction (via /graphify --mode deep or graphify extract), and build graph.json. The stored edge is:

   {"source": "auth_token_validation", "target": "login_session_verification",
    "relation": "semantically_similar_to", "confidence": "INFERRED", "source_file": "auth.md"}

   → 1 semantic edge present.

2. Run a full rebuild:

   graphify update .
   

3. Inspect graph.json:

   • Concept nodes auth_token_validation and login_session_verification: still present.
   • semantically_similar_to edges: 0 — evicted.

The graph grew overall (more AST nodes/edges), so the #479 shrink guard does not trigger — the loss is silent.

Expected

A full graphify update (AST-only) should preserve semantic edges owned by sources it re-extracts, the same way it already preserves semantic nodes. Only the AST-tier edges of a re-extracted source should be replaced.

Suggested fix

Make edge eviction provenance-aware, symmetric with node eviction: when reconciling a re-extracted source, evict only that source's AST-tier edges (e.g. gate the edge eviction on an _origin/tier marker, or only evict edges of a re-extracted source when a freshly extracted edge for that source is available to replace it). Semantic-tier edges of a re-extracted source should be preserved until a semantic re-extraction supersedes them — exactly the contract the surrounding comments already describe.

Workaround (for reference)

Re-union the committed semantic graph after every full update (via graphify merge-driver), which restores the evicted edges. This works but requires carrying a separate semantic snapshot and re-applying it everywhere a full rebuild runs.