Skip to content

fix(query): prevent generic symbol query flooding via multiplicity penalty and seed deduplication#1832

Closed
devcool20 wants to merge 1 commit into
Graphify-Labs:v8from
devcool20:fix-generic-symbol-flooding-1766
Closed

fix(query): prevent generic symbol query flooding via multiplicity penalty and seed deduplication#1832
devcool20 wants to merge 1 commit into
Graphify-Labs:v8from
devcool20:fix-generic-symbol-flooding-1766

Conversation

@devcool20

Copy link
Copy Markdown
Contributor

Description

This PR resolves Issue #1766, which causes query results to be flooded when codebases contain multiple generic/homonymous names (such as Next.js route handlers named GET or POST, or framework entrypoints like index or handler).

Previously, when a query matched a generic word (e.g., "get"), multiple distinct nodes sharing the same generic label (e.g., GET()) would receive the full _EXACT_MATCH_BONUS and get selected as query seeds. Running a BFS traversal from multiple generic nodes ended up pulling in massive portions of unrelated code, burying the actual specific target.


Proposed Changes

  1. Label Multiplicity Penalty (graphify/serve.py):

    • Computes and caches a map of normalized label counts (_label_counts_cache) on the graph G.
    • Divides the final matching score of any node by its label multiplicity (score /= multiplicity), ensuring generic framework convention names cannot easily overpower specific identifiers.
  2. Seed Label Deduplication (graphify/serve.py):

    • Adds a seen_labels tracking set in _pick_seeds to filter out duplicate labels, guaranteeing that only a single representative node gets seeded for any given homonymous label.

Validation & Testing

  • Scenario Mock Test: Verified using a Next.js route handler mock graph. Before the fix, querying "get users" seeded users_model, api_users_GET, and api_analytics_GET (flooding the BFS traversal). After the fix, the query correctly seeds only users_model and a single representative GET node (api_analytics_GET).
  • Regression Testing: Ran the full serve test suite (pytest tests/test_serve.py). All 80 unit tests passed successfully with 0 regressions.

@safishamsi

Copy link
Copy Markdown
Collaborator

Thanks @devcool20 — the seed-deduplication idea is the right fix and I've landed it in 6ca8604 (ships in 0.9.14), crediting you.

I kept the seed dedup (with two tweaks: the dedup key is normalized so GET/Get/get collapse together, and the per-term guarantee loop honors the same cap), but dropped the second mechanism — the per-label multiplicity penalty (score /= multiplicity). That penalty lives in _score_nodes, which shortest_path and explain also use to resolve endpoints, so dividing scores there silently reweights path/explain endpoint selection (out of scope for #1766 and able to flip which node a path anchors on). The dedup alone bounds the flood. I also added the regression tests the PR was missing. Appreciate the fix.

@safishamsi safishamsi closed this Jul 13, 2026
safishamsi added a commit that referenced this pull request Jul 13, 2026
Many nodes sharing one generic label (framework route handlers all
labelled GET/POST, a repeated handler) consumed every BFS seed slot, so
query traversal explored near-identical neighborhoods and buried the
actual target. Seed selection now dedups by normalized label (GET/Get/get
collapse together), keeping one representative per label, and the per-term
guarantee loop honors the same cap so it can't reintroduce a dupe.

Adopts @devcool20's seed-dedup from #1832 but drops that PR's second
mechanism — a per-label multiplicity penalty applied inside the shared
_score_nodes. That scorer also resolves shortest_path/explain endpoints,
so dividing scores there silently reweighted path/explain (out of scope
for #1766 and able to flip endpoint selection); the dedup alone bounds
the flood. Also normalizes the dedup key (the PR keyed on the raw label).
Adds the tests the PR was missing: dedup of homonymous labels,
case/diacritic normalization, per-term-guarantee cap, and a guard that
identical-label nodes still score equally in _score_nodes.

Co-Authored-By: devcool20 <devcool20@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants