Skip to content

query: relational verbs ("calls"/"callers"/"uses") hijack seed selection and bury the real identifier's callers (upstream Graphify-Labs#2507) #37

Description

@filipechagas

Upstream issue

Filed 2026-08-06: Graphify-Labs#2507 (same path fork #1 took for upstream Graphify-Labs#1682: upstream issue → fork spec → implementation → optional upstream PR). The 2026-08-06 sweep found no prior upstream issue covering demoting a relational-intent verb as a seed term.

The upstream issue references:

  1. Upstream issue 1445 / PR 1596 — the per-term seed guarantee. This is the amplifier: it turns the verb's junk prefix-match into a guaranteed seed slot that bypasses the gap cutoff.
  2. Issue 1900 / PR 1597 — the stopword precedent. Note carefully: PR 1597's tests affirmatively pinned "calls" as a kept content term, so this proposal challenges a deliberate upstream design point, not an oversight — the upstream issue must argue against that decision explicitly, not just around it.
  3. Open issue 2381 and open PR 2113 — the nearest live work (seed-path exclusion and per-term recovery bounding, respectively). Position term demotion as complementary: it fixes the cause (a non-content term holding a seed slot at all) where they bound the blast radius of a bad seed.

Framing hook for the upstream issue: _infer_context_filters already proves the system knows "calls" is relational intent for the query — phrasing A emitted Context: call (heuristic) — but that signal never reaches _query_terms/_pick_seeds, so the same word is simultaneously treated as an edge-filter directive and as an identifier competing for a guaranteed seat.

Related but not overlapping: open issue 2384 (node-kind priors) — adjacent seeding-quality work, distinct mechanism.

Problem Statement

When a user asks the semantic query command a natural-language impact question — the single most common question shape for an AI assistant consuming the graph — the answer is wrong or buried, and it fails silently: output renders, fills the token budget, and looks authoritative while containing zero of the correct nodes.

Measured 2026-08-06 on fork v0.9.34 against the api.lawnstarter.com graph @ d2d4bed7ae (46,406 nodes / 139,497 edges). Ground truth, grep-verified: ChargeCustomerService has exactly 3 production callers — ChargeCustomerCancellationFee, Jobs/Customer/BillingCharge, BalanceCustomerAccountService.

Phrasing Seeds chosen Result
"Who calls ChargeCustomerService?" ['ChargeCustomerService', '.callStoreWithAmount()'] + Context: call (heuristic) 0/3 callers; 15 nodes, all test noise from the junk seed's neighborhood, ~1,583 tok
"callers of ChargeCustomerService" ['ChargeCustomerService', '.it_ignores_initiator_for_non_ops_callers()'] 3/3 present but buried in 42 shown of 147 nodes, ~1,617 tok
"what code uses ChargeCustomerService to charge a customer" 5 seeds incl. .charge(), coder.md, a test method 0/3; 2,307-node explosion, 52 shown, ~1,600 tok

This is not a regression from the fork's PHP call-edge work: baseline v0.9.28 failed the same phrasings with the same .callStoreWithAmount() junk anchor. The fork's richer calls edge set just raises the stakes — good seeds now lead somewhere, so bad seeds cost more.

It is failure mode #2 of 3 that the PE-78803 spike's Recommendation 2 requires resolved before adoption (#1, the PHP calls gap, is fixed and shipped; #3, node-identity stub shadowing, is a separate problem — do not conflate).

Repro (~60s, no LLM cost — never run graphify update in ~/working/lawn/api itself; its graphify-out/ is the preserved v0.9.28 baseline):

cd ~/working/lawn/api && git worktree add --detach /tmp/api-sq d2d4bed7ae
cd /tmp/api-sq && graphify update . --no-cluster
graphify query "Who calls ChargeCustomerService?"   # observe the .callStoreWithAmount() seed

Root-cause hypothesis

⚠️ This is a code-read hypothesis corroborated by the observed seed lists. It has NOT been stepped through (no instrumented run of _score_query/_pick_seeds on the repro graph yet). Verify before building — the diagnosing-bugs loop on the repro graph is the intended first implementation step.

All line references are to graphify/serve.py on fork branch origin/v8.

  1. Relational verbs survive as content terms. _query_terms (line 253) drops _QUERY_STOPWORDS (line 221) — a question-word/filler list. "calls", "callers", "uses", "listens" are not in it, so they flow into scoring as if they were identifiers.
  2. "calls" prefix-matches a junk anchor. Scoring tiers (lines 275–277): exact 1000 / prefix 100 / substring 1.0, IDF-weighted (_compute_idf, line 281). The normalized label callstorewithamount literally starts with calls, so .callStoreWithAmount() takes the prefix tier for the term "calls" (line 559; amplified ×10 in the per-term singleton tier, line 583). Similarly "callers" substring-matches .it_ignores_initiator_for_non_ops_callers() — the singleton score also accumulates substring hits (line 590), so a junk node can win a term's best_seed_by_term slot on substring alone.
  3. The BFS path drift on vague Chinese queries: 40-node corpus, 12 unrelated nodes returned for "rate_limit_check 怎么用?" Graphify-Labs/graphify#1445 per-term guarantee then locks the junk in. _pick_seeds (line 633) first applies the top-score + 20%-gap window with label dedup (query: homonymous generic symbols (e.g. many GET() route handlers) all get seeded, flooding results — related to #1303 Graphify-Labs/graphify#1766), then — lines 702–722 — appends best_seed_by_term[term] for every distinct query term, bypassing the gap cutoff entirely. The guarantee that protects real multi-term queries from starvation also guarantees the verb's junk anchor a seed slot no matter how dominant the real identifier's score was.
  4. BFS floods the budget from the junk seed. The pipeline (_query_graph_text, line 1090: _query_terms_score_query_pick_seeds at line 1108 → _bfs/_dfs_subgraph_to_text, default 2,000-token budget) then spends the budget on the junk seed's test-file neighborhood.
  5. The sharp asymmetry: _infer_context_filters (line 789, _CONTEXT_HINTS line 727) already recognizes "calls"/"callers"/"uses"-adjacent words as relational intent — phrasing A emitted Context: call (heuristic). But that recognition only filters traversal edges; it never demotes the same word as a seed term. A term consumed as relation intent arguably should not also compete as an identifier — or at minimum should lose the per-term guarantee.

Constraints on any fix

Acceptance Criteria

Graded on the repro graph (api.lawnstarter.com @ d2d4bed7ae, fork build reinstalled from current v8 head first):

  1. "Who calls ChargeCustomerService?" → output contains all 3 known callers (ChargeCustomerCancellationFee, Jobs/Customer/BillingCharge, BalanceCustomerAccountService), within the default 2,000-token budget, without the .callStoreWithAmount() (or equivalent verb-prefix junk) seed.
  2. "callers of ChargeCustomerService" → all 3 callers present and not buried: they appear in the shown portion of the output, and the shown node set is not dominated by an unrelated seed's neighborhood (the ~147-node / 42-shown haystack shrinks materially).
  3. "what code uses ChargeCustomerService to charge a customer" → all 3 callers found; no multi-thousand-node explosion (the 2,307-node traversal is the failure bar), within budget.
  4. No-regression check: graphify explain for "What listens to CustomerWasCharged?" — passed pre-fix at ~417 tokens — still returns the correct listener(s) at comparable token cost.
  5. Full existing pytest suite green, explicitly including every test cited in Constraints above.
  6. New red-→-green tests derived from the 3-phrasing table (the fork has a pytest suite; the table converts directly into fixtures — a corpus with a ChargeX-like service, a verb-prefix decoy like callStoreWithAmount, and 3 known callers).

"Low noise" operationally: the 3 correct callers are within the rendered (shown) node list, and no single junk seed contributes the majority of shown nodes.

Suggested labels

  • needs-triage — now: the hypothesis is unverified and the new upstream issue is not yet filed; per docs/agents/triage-labels.md this is "maintainer needs to evaluate".
  • Promote to ready-for-agent once (a) the hypothesis is confirmed by an instrumented step-through on the repro graph, (b) the new upstream issue is filed and linked above, and (c) a Solution/Implementation Decisions spec is added — the same bar fork issue PHP: resolve instance-method calls edges from typed receivers (upstream Graphify-Labs#1682) #1 met before it carried that label.

Prior art / references

Metadata

Metadata

Assignees

No one assigned

    Labels

    ready-for-agentFully specified, ready for an AFK agent

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions