Skip to content

build_merge(prune_sources=...) matches raw strings, so --update never prunes deleted files and reports "already clean" #3153

Description

@maxim1976

build_merge(prune_sources=...) compares raw strings, so --update never prunes deleted files — and reports "already clean" while doing it.

Version: graphifyy 0.8.14, macOS (darwin), Python 3.13

What happens

I deleted one tracked file (dash/booking/views.py) and ran the --update flow from SKILL.md. detect_incremental correctly reported the deletion, but the nodes survived the merge and the run printed:

[graphify] 1 source file(s) deleted since last run — no matching nodes or edges in graph, already clean.
[graphify update] Merged: 2318 nodes, 4723 edges

The graph was not clean. Two nodes (booking_views_placeholder, dash_booking_views_py) were still present, pointing at a file that no longer exists. Because the message claims success, this is silent — every deletion leaves ghost nodes behind and nothing surfaces it. In a long-lived graph the nodes accumulate and keep showing up in explain, query, and God Nodes.

Root cause

Two independent mismatches, stacked.

1. Path form. detect_incremental returns deleted_files as absolute paths, but nodes store source_file relative to the scan root. build_merge prunes by plain string equality:

prune_set = set(prune_sources)
to_remove = [n for n, d in G.nodes(data=True) if d.get("source_file") in prune_set]

So '/Users/me/proj/dash/booking/views.py' is compared against the stored 'dash/booking/views.py' and never matches. build_merge already takes a root= argument that normalises absolute source_file values — but it applies it to new_chunks only, never to prune_sources.

2. Path case (macOS/Windows). The obvious fix — Path(deleted).resolve().relative_to(scan_root.resolve()) — also fails whenever .graphify_root was written with different case than the path detect returns. Here .graphify_root held /Users/maxim/documents/... while detect returned /Users/maxim/Documents/.... Both are valid on a case-insensitive filesystem, but Path.resolve() does not normalise case and os.path.normcase is a no-op on POSIX, so relative_to raises ValueError. If that's caught and skipped, the deletion is dropped a second time, just as silently.

Repro

from graphify.build import build_merge
# graph.json contains a node with source_file == 'pkg/gone.py'
G = build_merge([{'nodes': [], 'edges': []}],
                graph_path='graphify-out/graph.json',
                prune_sources=['/abs/path/to/proj/pkg/gone.py'])
assert 'the_node_id' not in G   # fails; prints "already clean"

Suggested fix

  1. In build_merge, normalise prune_sources the same way root= already normalises new_chunks, and match against both the absolute and root-relative spelling rather than one raw form.
  2. Case-fold the prefix comparison on sys.platform in ('darwin', 'win32'). os.path.normcase is not sufficient — it does nothing on POSIX.
  3. Don't print "already clean" purely because the string comparison matched nothing. When prune_sources is non-empty and zero nodes were removed, that is the failure signature, not a clean graph — it deserves a warning.

Working version of (1) and (2):

ci = sys.platform in ('darwin', 'win32')
key = (lambda s: s.lower()) if ci else (lambda s: s)
root_s = str(Path(root).resolve()).rstrip(os.sep) + os.sep
forms = set()
for p in prune_sources:
    ap = str(Path(p).resolve())
    forms.add(str(p)); forms.add(ap)
    if key(ap).startswith(key(root_s)):
        forms.add(ap[len(root_s):])       # case-insensitive prefix strip

Verified against the real case: prune set becomes ['/Users/.../dash/booking/ghost_probe.py', 'dash/booking/ghost_probe.py'] and build_merge then reports Pruned 1 node(s) with the node genuinely gone.

Also worth patching

SKILL.md's --update section passes prune_sources=deleted or None with detect's raw absolute paths, so the documented workflow hits this on every deletion. Normalising there (or fixing build_merge) closes it for skill users.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions