[extract] Adaptive deeper file stem on collision (opt-in via resolve_stems_for_corpus)

[ibrahimyuecel]

[extract] Auto-deepen file stem on parent collision

Summary

_file_stem(path) returns f"{parent_dir}.{file_stem}" — only one parent
segment. In feature-based architectures (NestJS, Next.js, Angular, Django apps,
Go cmd/X, Rust workspaces) the same parent name (index.ts, routes.py,
schema.go, mod.rs) appears in many directories.

This causes node ID collisions: dozens of files share the same stem, and
the _make_id(stem, name) IDs collapse them. God nodes report shows
inflated degrees, communities mix unrelated features, and queries return
wrong-feature matches.

This PR auto-deepens the stem (walks up additional parents) when a collision
is detected during initial extraction, with no API change to consumers.

Why this matters

Tested on a SharedModel monorepo (~3,800 TypeScript files):

Stem strategy	Distinct stems	Stems with multiple files	Affected files
Current (1 parent)	3,464	66	271
Deepen to 2 parents	—	36	~150
Deepen to 3 parents	—	3	6

Real impact:

• 13 different dtos/index.ts files all map to stem dtos.index
• 11 different queries/index.ts collapse together
• 11 hooks/index.ts, 10 components/index.ts, 9 constants/index.ts ...
• All barrel exports across all features become indistinguishable

When a query asks for studyRoutes (defined in study/constants/routes.ts,
re-exported through study/constants/index.ts), the barrel resolver finds
(constants.index, studyRoutes) — but 9 different barrels claim the same
stem, so the edge attribution is non-deterministic.

Proposed change

Strategy A: Adaptive deepening (recommended)

In extract.py, two-pass:

1. Pass 1 — collect all file paths, compute initial 1-parent stems.
2. For each stem with >1 file, deepen those files' stems by walking up one
   more parent until they're unique (or hit a hard cap of 5 levels).

def _resolve_stems(paths: list[Path]) -> dict[Path, str]:
    """Compute file stems, auto-deepening on collision."""
    stem_to_paths = defaultdict(list)
    for p in paths:
        stem_to_paths[_initial_stem(p)].append(p)

    out: dict[Path, str] = {}
    for initial_stem, group in stem_to_paths.items():
        if len(group) == 1:
            out[group[0]] = initial_stem
            continue
        # Multiple files share this stem — find deepest path that distinguishes
        for p in group:
            depth = 2
            while depth <= 5:
                candidate = _stem_at_depth(p, depth)
                if all(_stem_at_depth(other, depth) != candidate or other == p
                       for other in group):
                    out[p] = candidate
                    break
                depth += 1
            else:
                # Hard fallback: include hash suffix
                out[p] = f"{initial_stem}.{hash(str(p)) & 0xFFFF:04x}"
    return out


def _stem_at_depth(path: Path, depth: int) -> str:
    """Return file stem joined with `depth` parent dir names."""
    segs = []
    cur = path
    for _ in range(depth):
        if cur.parent.name and cur.parent.name not in (".", ""):
            segs.append(cur.parent.name)
            cur = cur.parent
        else:
            break
    segs.reverse()
    segs.append(path.stem)
    return ".".join(segs)

Strategy B: Always deepen to N (simpler, breaking)

Just change _file_stem() to always include 2 or 3 parents. Backward-compat
breaking — old graphs become unrecognizable, all node IDs shift.

Strategy C: Opt-in flag (--deep-stem)

Add CLI flag to enable adaptive deepening. Default behaviour unchanged. Most
conservative.

Recommended: Strategy A (zero config, backward-compat for non-colliding
stems, only colliding files get longer IDs).

Backward compatibility

• Strategy A: only files in collision groups get longer stems. Existing graphs
  with no collisions are byte-identical.
• Strategy B: breaking. Requires major version bump and migration path.
• Strategy C: fully backward-compat (opt-in).

Test fixture

tests/fixtures/stem_collision/
├── feature_a/
│   ├── routes.ts
│   └── index.ts
├── feature_b/
│   ├── routes.ts
│   └── index.ts
└── feature_c/
    ├── routes.ts
    └── index.ts

Expected (Strategy A):

• feature_a/routes.ts → stem feature_a.routes (deepened, was feature_a.routes already — coincidence)
• All index.ts files → feature_a.index, feature_b.index, feature_c.index

Without this PR:

• All routes.ts collapse to stem feature_a.routes, feature_b.routes, feature_c.routes (these don't collide today by luck of unique parents)
• All index.ts collapse to same index stem — the bug

Out of scope

• Choosing between the three strategies (community discussion preferred)
• Handling Git rename/move (stem stability across renames is a separate concern)
• ID stability across runs when a new file enters a collision group (Strategy A
  changes group members' stems when collision set changes — annoying for
  incremental update if someone deletes a file)

Tested against

SharedModel monorepo:

• 271 files → 6 files affected after Strategy A (depth-3 max)
• 66 distinct collision groups → 3 unresolvable (likely truly identical paths)
• God nodes report no longer shows synthesized "dtos.index" hub
• Community detection separates per-feature concerns instead of mixing them

[jippi]

Hit collision-driven node confusion on a 1,873-file SvelteKit codebase that this PR would address. Adding our cases for evidence.

Symptom on our side: 33 self-loops in the graph after extraction. Sampling them, several patterns trace back to _make_id(stem, name) collisions where stem is too shallow:

• LinkDomain (Sequelize model) has a [shares_data_with] self-loop because the file node and the class node end up with the same _make_id after _file_stem truncates to one parent. The model file is src/lib/server/models/LinkDomain.model.ts (file_stem models.LinkDomain.model), and the exported class is LinkDomain. The class's _make_id(stem, name) and the file's _make_id(str(path)) collide at the canonical-ID layer.

• Comment model exhibits the same pattern — models.Comment.model stem, Comment class. Self-loop emitted.

• Multiple [calls] self-loops on canonical-rule files that share a parent dir name with sibling files of the same stem.

Repo-wide: ~30 of 33 self-loops are likely this bug class (3 are valid SQL FK self-references on tags, publishers, work_collections).

This PR's deepening strategy (auto-walk to 2+ parents on collision) would resolve all of them. The deepening is a strict superset of fixes — once stems are unique, the canonical-ID collisions go away and the spurious self-loops with them.

One observation worth pinning down: the models/*.model.ts pattern is interesting because the file already encodes its kind in the basename (*.model.ts). For Sequelize/TypeORM/Mongoose codebases this pattern is dense — dozens of <Name>.model.ts files with classes named <Name> inside. Even the deepened stem (models.LinkDomain.model) collides with the class id (LinkDomain qualified to the same stem) at _make_id. May be worth verifying the deepening alone resolves it, vs. needing the file-vs-class id namespacing change too.

(Same methodology note as #709 — wipe graphify-out/ fully before measuring; incremental rebuilds preserve stale data.)

[jippi]

Considering we're on v7, doing breaking changes in a v8 at any time for a stable long term solution seems preferable over more config knobs and flags