You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Filed 2026-08-06: Graphify-Labs#2507 (same path fork #1 took for upstream Graphify-Labs#1682: upstream issue → fork spec → implementation → optional upstream PR). The 2026-08-06 sweep found no prior upstream issue covering demoting a relational-intent verb as a seed term.
The upstream issue references:
Upstream issue 1445 / PR 1596 — the per-term seed guarantee. This is the amplifier: it turns the verb's junk prefix-match into a guaranteed seed slot that bypasses the gap cutoff.
Issue 1900 / PR 1597 — the stopword precedent. Note carefully: PR 1597's tests affirmatively pinned "calls" as a kept content term, so this proposal challenges a deliberate upstream design point, not an oversight — the upstream issue must argue against that decision explicitly, not just around it.
Open issue 2381 and open PR 2113 — the nearest live work (seed-path exclusion and per-term recovery bounding, respectively). Position term demotion as complementary: it fixes the cause (a non-content term holding a seed slot at all) where they bound the blast radius of a bad seed.
Framing hook for the upstream issue:_infer_context_filters already proves the system knows "calls" is relational intent for the query — phrasing A emitted Context: call (heuristic) — but that signal never reaches _query_terms/_pick_seeds, so the same word is simultaneously treated as an edge-filter directive and as an identifier competing for a guaranteed seat.
Related but not overlapping: open issue 2384 (node-kind priors) — adjacent seeding-quality work, distinct mechanism.
Problem Statement
When a user asks the semantic query command a natural-language impact question — the single most common question shape for an AI assistant consuming the graph — the answer is wrong or buried, and it fails silently: output renders, fills the token budget, and looks authoritative while containing zero of the correct nodes.
Measured 2026-08-06 on fork v0.9.34 against the api.lawnstarter.com graph @ d2d4bed7ae (46,406 nodes / 139,497 edges). Ground truth, grep-verified: ChargeCustomerService has exactly 3 production callers — ChargeCustomerCancellationFee, Jobs/Customer/BillingCharge, BalanceCustomerAccountService.
3/3 present but buried in 42 shown of 147 nodes, ~1,617 tok
"what code uses ChargeCustomerService to charge a customer"
5 seeds incl. .charge(), coder.md, a test method
0/3; 2,307-node explosion, 52 shown, ~1,600 tok
This is not a regression from the fork's PHP call-edge work: baseline v0.9.28 failed the same phrasings with the same .callStoreWithAmount() junk anchor. The fork's richer calls edge set just raises the stakes — good seeds now lead somewhere, so bad seeds cost more.
It is failure mode #2 of 3 that the PE-78803 spike's Recommendation 2 requires resolved before adoption (#1, the PHP calls gap, is fixed and shipped; #3, node-identity stub shadowing, is a separate problem — do not conflate).
Repro (~60s, no LLM cost — never run graphify update in ~/working/lawn/api itself; its graphify-out/ is the preserved v0.9.28 baseline):
⚠️This is a code-read hypothesis corroborated by the observed seed lists. It has NOT been stepped through (no instrumented run of _score_query/_pick_seeds on the repro graph yet). Verify before building — the diagnosing-bugs loop on the repro graph is the intended first implementation step.
All line references are to graphify/serve.py on fork branch origin/v8.
Relational verbs survive as content terms._query_terms (line 253) drops _QUERY_STOPWORDS (line 221) — a question-word/filler list. "calls", "callers", "uses", "listens" are not in it, so they flow into scoring as if they were identifiers.
"calls" prefix-matches a junk anchor. Scoring tiers (lines 275–277): exact 1000 / prefix 100 / substring 1.0, IDF-weighted (_compute_idf, line 281). The normalized label callstorewithamount literally starts with calls, so .callStoreWithAmount() takes the prefix tier for the term "calls" (line 559; amplified ×10 in the per-term singleton tier, line 583). Similarly "callers" substring-matches .it_ignores_initiator_for_non_ops_callers() — the singleton score also accumulates substring hits (line 590), so a junk node can win a term's best_seed_by_term slot on substring alone.
BFS floods the budget from the junk seed. The pipeline (_query_graph_text, line 1090: _query_terms → _score_query → _pick_seeds at line 1108 → _bfs/_dfs → _subgraph_to_text, default 2,000-token budget) then spends the budget on the junk seed's test-file neighborhood.
The sharp asymmetry:_infer_context_filters (line 789, _CONTEXT_HINTS line 727) already recognizes "calls"/"callers"/"uses"-adjacent words as relational intent — phrasing A emitted Context: call (heuristic). But that recognition only filters traversal edges; it never demotes the same word as a seed term. A term consumed as relation intent arguably should not also compete as an identifier — or at minimum should lose the per-term guarantee.
No blanket stopwording of "calls"/"callers"/"uses". These can be legitimate identifiers in some corpus (a calls table, a callers() method); adding them to _QUERY_STOPWORDS is the naive fix that regresses someone. Any demotion must be conditional (e.g. only when the term was consumed as relational intent by _resolve_context_filters, or only when other terms produced strong identifier matches).
Protected behavior is defined by the existing tests around _pick_seeds and _score_query, all on origin/v8:
tests/test_serve.py:385 — non-English (German) query seeds a content node, not heading noise
tests/test_serve.py:1119, 1128, 1138 — property tests pinning _score_query's ranked output and best_seed_by_term to the legacy per-token semantics (via the _reference_best_seed_by_term oracle at line 1054)
tests/bench_query_scoring.py — non-CI microbenchmark that asserts equality of ranked/best_seed_by_term/seeds between legacy and single-pass paths before timing; a fix that changes seeding semantics must update its equality baseline deliberately, not accidentally
The fix lands on a branch off v8. The graph is rebuilt fresh in a throwaway worktree for grading (see repro); the preserved v0.9.28 baseline graph in ~/working/lawn/api/graphify-out/ must not be touched.
Acceptance Criteria
Graded on the repro graph (api.lawnstarter.com @ d2d4bed7ae, fork build reinstalled from current v8 head first):
"Who calls ChargeCustomerService?" → output contains all 3 known callers (ChargeCustomerCancellationFee, Jobs/Customer/BillingCharge, BalanceCustomerAccountService), within the default 2,000-token budget, without the .callStoreWithAmount() (or equivalent verb-prefix junk) seed.
"callers of ChargeCustomerService" → all 3 callers present and not buried: they appear in the shown portion of the output, and the shown node set is not dominated by an unrelated seed's neighborhood (the ~147-node / 42-shown haystack shrinks materially).
"what code uses ChargeCustomerService to charge a customer" → all 3 callers found; no multi-thousand-node explosion (the 2,307-node traversal is the failure bar), within budget.
No-regression check:graphify explain for "What listens to CustomerWasCharged?" — passed pre-fix at ~417 tokens — still returns the correct listener(s) at comparable token cost.
Full existing pytest suite green, explicitly including every test cited in Constraints above.
New red-→-green tests derived from the 3-phrasing table (the fork has a pytest suite; the table converts directly into fixtures — a corpus with a ChargeX-like service, a verb-prefix decoy like callStoreWithAmount, and 3 known callers).
"Low noise" operationally: the 3 correct callers are within the rendered (shown) node list, and no single junk seed contributes the majority of shown nodes.
Suggested labels
needs-triage — now: the hypothesis is unverified and the new upstream issue is not yet filed; per docs/agents/triage-labels.md this is "maintainer needs to evaluate".
Promote to ready-for-agent once (a) the hypothesis is confirmed by an instrumented step-through on the repro graph, (b) the new upstream issue is filed and linked above, and (c) a Solution/Implementation Decisions spec is added — the same bar fork issue PHP: resolve instance-method calls edges from typed receivers (upstream Graphify-Labs#1682) #1 met before it carried that label.
Follow-up spike (Finding 5 is the semantic-query evidence): brain/eng/docs/spikes/2026-08-06-PE-78803-re-run-the-graphify-poc-gate-with-the-lawnstarter-fork.md.
Upstream issue
Filed 2026-08-06: Graphify-Labs#2507 (same path fork #1 took for upstream Graphify-Labs#1682: upstream issue → fork spec → implementation → optional upstream PR). The 2026-08-06 sweep found no prior upstream issue covering demoting a relational-intent verb as a seed term.
The upstream issue references:
Framing hook for the upstream issue:
_infer_context_filtersalready proves the system knows "calls" is relational intent for the query — phrasing A emittedContext: call (heuristic)— but that signal never reaches_query_terms/_pick_seeds, so the same word is simultaneously treated as an edge-filter directive and as an identifier competing for a guaranteed seat.Related but not overlapping: open issue 2384 (node-kind priors) — adjacent seeding-quality work, distinct mechanism.
Problem Statement
When a user asks the semantic
querycommand a natural-language impact question — the single most common question shape for an AI assistant consuming the graph — the answer is wrong or buried, and it fails silently: output renders, fills the token budget, and looks authoritative while containing zero of the correct nodes.Measured 2026-08-06 on fork v0.9.34 against the api.lawnstarter.com graph @
d2d4bed7ae(46,406 nodes / 139,497 edges). Ground truth, grep-verified:ChargeCustomerServicehas exactly 3 production callers —ChargeCustomerCancellationFee,Jobs/Customer/BillingCharge,BalanceCustomerAccountService.['ChargeCustomerService', '.callStoreWithAmount()']+Context: call (heuristic)['ChargeCustomerService', '.it_ignores_initiator_for_non_ops_callers()'].charge(),coder.md, a test methodThis is not a regression from the fork's PHP call-edge work: baseline v0.9.28 failed the same phrasings with the same
.callStoreWithAmount()junk anchor. The fork's richercallsedge set just raises the stakes — good seeds now lead somewhere, so bad seeds cost more.It is failure mode #2 of 3 that the PE-78803 spike's Recommendation 2 requires resolved before adoption (#1, the PHP calls gap, is fixed and shipped; #3, node-identity stub shadowing, is a separate problem — do not conflate).
Repro (~60s, no LLM cost — never run
graphify updatein~/working/lawn/apiitself; itsgraphify-out/is the preserved v0.9.28 baseline):Root-cause hypothesis
All line references are to
graphify/serve.pyon fork branchorigin/v8._query_terms(line 253) drops_QUERY_STOPWORDS(line 221) — a question-word/filler list. "calls", "callers", "uses", "listens" are not in it, so they flow into scoring as if they were identifiers._compute_idf, line 281). The normalized labelcallstorewithamountliterally starts withcalls, so.callStoreWithAmount()takes the prefix tier for the term "calls" (line 559; amplified ×10 in the per-term singleton tier, line 583). Similarly "callers" substring-matches.it_ignores_initiator_for_non_ops_callers()— the singleton score also accumulates substring hits (line 590), so a junk node can win a term'sbest_seed_by_termslot on substring alone._pick_seeds(line 633) first applies the top-score + 20%-gap window with label dedup (query: homonymous generic symbols (e.g. many GET() route handlers) all get seeded, flooding results — related to #1303 Graphify-Labs/graphify#1766), then — lines 702–722 — appendsbest_seed_by_term[term]for every distinct query term, bypassing the gap cutoff entirely. The guarantee that protects real multi-term queries from starvation also guarantees the verb's junk anchor a seed slot no matter how dominant the real identifier's score was._query_graph_text, line 1090:_query_terms→_score_query→_pick_seedsat line 1108 →_bfs/_dfs→_subgraph_to_text, default 2,000-token budget) then spends the budget on the junk seed's test-file neighborhood._infer_context_filters(line 789,_CONTEXT_HINTSline 727) already recognizes "calls"/"callers"/"uses"-adjacent words as relational intent — phrasing A emittedContext: call (heuristic). But that recognition only filters traversal edges; it never demotes the same word as a seed term. A term consumed as relation intent arguably should not also compete as an identifier — or at minimum should lose the per-term guarantee.Constraints on any fix
tests/test_serve.py:810(test_pick_seeds_diversity_recovers_starved_term).callstable, acallers()method); adding them to_QUERY_STOPWORDSis the naive fix that regresses someone. Any demotion must be conditional (e.g. only when the term was consumed as relational intent by_resolve_context_filters, or only when other terms produced strong identifier matches)._pick_seedsand_score_query, all onorigin/v8:tests/test_serve.py:772–807— core_pick_seedsgap-window behavior (graphify query: seed scoring returns low-relevance nodes, causing multi-call token cascade Graphify-Labs/graphify#897)tests/test_serve.py:810— BFS path drift on vague Chinese queries: 40-node corpus, 12 unrelated nodes returned for "rate_limit_check 怎么用?" Graphify-Labs/graphify#1445 starvation recovery (see above)tests/test_serve.py:135, 180— query: single generic term exact-matching a short leaf label hijacks seed selection in multi-term queries Graphify-Labs/graphify#1602 coverage scaling; single-identifier queries stay byte-identicaltests/test_serve.py:845, 862, 873— query: homonymous generic symbols (e.g. many GET() route handlers) all get seeded, flooding results — related to #1303 Graphify-Labs/graphify#1766 homonymous-label seed dedup, incl. that the per-term guarantee cannot reintroduce a deduped generic labeltests/test_serve.py:385— non-English (German) query seeds a content node, not heading noisetests/test_serve.py:1119, 1128, 1138— property tests pinning_score_query's ranked output andbest_seed_by_termto the legacy per-token semantics (via the_reference_best_seed_by_termoracle at line 1054)tests/bench_query_scoring.py— non-CI microbenchmark that asserts equality of ranked/best_seed_by_term/seeds between legacy and single-pass paths before timing; a fix that changes seeding semantics must update its equality baseline deliberately, not accidentallyv8. The graph is rebuilt fresh in a throwaway worktree for grading (see repro); the preserved v0.9.28 baseline graph in~/working/lawn/api/graphify-out/must not be touched.Acceptance Criteria
Graded on the repro graph (api.lawnstarter.com @
d2d4bed7ae, fork build reinstalled from currentv8head first):ChargeCustomerCancellationFee,Jobs/Customer/BillingCharge,BalanceCustomerAccountService), within the default 2,000-token budget, without the.callStoreWithAmount()(or equivalent verb-prefix junk) seed.graphify explainfor "What listens to CustomerWasCharged?" — passed pre-fix at ~417 tokens — still returns the correct listener(s) at comparable token cost.ChargeX-like service, a verb-prefix decoy likecallStoreWithAmount, and 3 known callers)."Low noise" operationally: the 3 correct callers are within the rendered (shown) node list, and no single junk seed contributes the majority of shown nodes.
Suggested labels
needs-triage— now: the hypothesis is unverified and the new upstream issue is not yet filed; perdocs/agents/triage-labels.mdthis is "maintainer needs to evaluate".ready-for-agentonce (a) the hypothesis is confirmed by an instrumented step-through on the repro graph, (b) the new upstream issue is filed and linked above, and (c) a Solution/Implementation Decisions spec is added — the same bar fork issue PHP: resolve instance-method calls edges from typed receivers (upstream Graphify-Labs#1682) #1 met before it carried that label.Prior art / references
brain/eng/docs/spikes/2026-08-06-PE-78803-re-run-the-graphify-poc-gate-with-the-lawnstarter-fork.md.