feat(v0.19): make --spi pay off — framework metadata + retrieval boost for Hono/Fastify/tRPC/Prisma#129
Conversation
…t for Hono/Fastify/tRPC/Prisma
Two complementary pieces that turn the v0.14-v0.18 substrate work into actual end-user value:
## 1. Confirmed: framework metadata flows end-to-end with --spi
Tightened tests/unit/generate-spi-flag.test.ts to assert that graph.json nodes produced via --spi carry the framework_role / route_path / node_kind / framework fields. The serializer (toJson in src/pipeline/export.ts) already spreads all node attributes via ...attributes — my earlier claim that 'fields are dropped' was wrong (the test assertion was checking for the wrong whitespace formatting). Verified by debug test that the express handler node in graph.json contains framework:'express', framework_role:'express_route', node_kind:'route', route_path:'/users'.
## 2. Retrieval boost for the v0.17 substrates (Hono / Fastify / tRPC / Prisma)
The existing frameworkBoostForNode() in src/runtime/retrieve.ts only handled express/redux/reactRouter/nest/next. v0.17 added 4 more substrates but the retrieval engine ignored their framework_role values, so questions like 'show me hono routes' got no boost over plain label matching.
Now wired:
- New profile slots: hono / fastify / trpc / prisma
- New intents: pluginIntent / procedureIntent / queryIntent / mutationIntent / subscriptionIntent / modelIntent
- Boost rules:
* hono_route: 4 with routeIntent, 1.5 otherwise
* hono_middleware: 2.5 with middlewareIntent, 1 otherwise
* fastify_route: 4 with routeIntent, 1.5 otherwise
* fastify_plugin: 2.5 with pluginIntent or middlewareIntent
* trpc_procedure_query: 3.5 with queryIntent or procedureIntent, 1.5 otherwise
* trpc_procedure_mutation: 3.5 with mutationIntent or procedureIntent, 1.5 otherwise
* trpc_procedure_subscription: 3.5 with subscriptionIntent or procedureIntent, 1.5 otherwise
* trpc_router: 2 with routeIntent or procedureIntent, 0.75 otherwise
* prisma_client: 3 with modelIntent, 1.5 otherwise
- activeFrameworksForProfile registers the new framework names so the framework-compatible filter recognises hono/fastify/trpc/prisma
- Bonus: express boost is now suppressed when hono/fastify are explicit (prevents misrouting Hono questions to Express)
## What this means for tokens / benchmarks
Same number of nodes per retrieval, but framework-shaped questions ('show me hono routes', 'what tRPC mutations exist', 'find the Prisma client') now favour the structurally-correct nodes via framework_role boost. Concretely: fewer plain function/class matches push the substrate-correct nodes off the budget cutoff, which lets the user-visible context_pack response surface the right nodes inside the budget instead of bumping it.
Whether this moves the benchmark numbers depends on the benchmark queries — for framework-shaped queries it should; for pure code-comprehension queries it shouldn't (no boost fires).
## Tests
- 5 new tests in tests/unit/retrieve-framework-boost-v0-19.test.ts: Hono/Fastify/tRPC/Prisma boost paths + a negative case (unrelated question gets no boost)
- Updated the v0.18 test to actually assert framework_role + route_path flow into graph.json
- 1817/1817 pass, typecheck + build clean
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughThis PR extends the framework-aware retrieval system to recognize and boost framework-specific nodes for Hono, Fastify, tRPC, and Prisma. The runtime retrieval module gains new profile types, intent signals, framework selection logic with precedence rules, and dedicated node boost scoring. Tests validate SPI attribute propagation and framework-specific retrieval behavior. ChangesFramework-Aware Retrieval Boost Extension
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
…ethod / slice_name / procedure_name (#137) * feat(#133): metadata-aware framework boost in retrieve — uses route_path / http_method / slice_name / procedure_name / mount_path Generalises framework-aware retrieval from 'role-only' boost to 'role + metadata' boost. PR #129 added boost rules per framework_role. This PR uses the structural metadata each role carries to add targeted substring-match boosts. ## Substrate metadata audit + plumbing fix Discovered while writing tests: Express was tagging route_path but NOT http_method on framework_metadata. The Hono detector already did this; Express was behind. Patched framework-express.ts emitRouteForCall to co-attach http_method alongside route_path when the path is statically known. Dynamic-path contract (no metadata) preserved. ## New boost rules in frameworkBoostForNode When the question contains a substring matching the node's metadata, add a targeted boost on top of the existing role-based boost: - route_path match (>=3 chars, substring): +3 boost - http_method match (case-insensitive): +1.5 boost - mount_path match (>=3 chars, substring): +1.5 boost - slice_name match (>=2 chars, substring): +2 boost - procedure_name match (>=2 chars, substring): +2 boost - router_name match (>=2 chars, substring): +1 boost ## Why these substrings + thresholds Substrings, not exact match: users phrase queries informally ('the GET /users/:id route', 'how does cancelOrder work', 'auth slice'). Substring match catches these without requiring the question to syntactically own the path. Minimum-length guards prevent noise: a 1-char route_path like '/' would match every English sentence. 3-char minimum on paths + 2-char minimum on names is the threshold I found empirically; it ignores '/' and 't' / 'a' style names while keeping real ones. ## Plumbing - FrameworkNodeMetadata interface added (route_path | http_method | mount_path | slice_name | procedure_name | router_name) - frameworkMetadataFromAttributes(attributes) extracts the fields from a graph node attribute bag - frameworkBoostForNode signature extended: now takes (profile, nodeKind, role, metadata, questionLower) - Both call sites updated (scoredNodeFromGraphEntry path + seedCandidates path) ## Tests - 5 new tests in tests/unit/retrieve-metadata-boost-133.test.ts: route_path match, http_method match (POST vs GET differentiation), slice_name match (auth vs counter), procedure_name match (cancelOrder vs getUser), negative case (unrelated query gets no metadata boost) - 3 existing Express test fixtures updated to assert http_method alongside route_path on framework_metadata - 1822/1822 tests pass, typecheck + build clean ## Benchmark re-run Re-ran the #130 fixture with #133 enabled. spi total: 946 tokens (unchanged). legacy total: 1160 (varies ~10% run-to-run on this small fixture). The substantive improvement is in CORRECTNESS — the unit tests prove the boost differentiates correctly between nodes that previously tied (POST vs GET routes on /users, auth vs counter slices, cancelOrder vs getUser procedures). * fix(#133): metadata-only seeding + word-boundary http_method match (CodeRabbit) Two valid findings from CodeRabbit on PR #137: 1) MAJOR — metadata-only matches couldn't enter the seed set. Code only pushed seeds when score.total > 0, meaning a handler named 'h()' tagged with route_path: '/orders/:id' was invisible to lexical retrieval even when the question literally contained that path. Fix: compute framework boost BEFORE the gate, and allow seeding when either lexical score OR metadata boost is positive. Metadata-only seeds get evidence tier >= 1. 2) HTTP-verb substring match was too greedy — 'GET' would match 'budget', 'forget', 'target' etc. Fix: switched from .includes(verb) to a word-boundary regex \\b${verb}\\b. Tests: - New regression: handler 'h()' with no label overlap is now seeded when route_path matches the question. Pre-fix this node was invisible. - New regression: 'budget' in the question does NOT trigger the http_method GET boost; literal 'GET' does. - 1824/1824 pass. * fix(#133): don't double-count frameworkBoost in async retrieval + escape regex metacharacters (CodeRabbit) Two valid follow-ups from CodeRabbit on PR #137: 1) MAJOR — frameworkBoost was double-counted in the async retrieval path. lexicalScoresById holds match_score values from retrieveContext, which already bake in frameworkBoost. The downstream merge then added candidate.frameworkBoost AGAIN, inflating lexical winners by an extra +5-8 points. Fix: gate frameworkBoost addition on lexicalScore === 0, so it only contributes to semantic-only candidates. Lexical winners keep their already-boosted score; semantic-only candidates with metadata matches still get the boost. 2) Defensive regex escape on http_method. Even though HTTP methods are letters-only in practice, escaping metacharacters before constructing the RegExp prevents future regression if metadata.http_method ever leaks user-controlled or otherwise unexpected content. 1824/1824 tests still pass.
…'what's new' callout + test recipe (#140) * docs(readme): v0.20 refresh — framework substrate audit + npm video fallback + 'what's new' callout + test recipe Honest answers to all 4 user concerns: 1) README was stale for v0.18-v0.20. Updated: - Top-of-file 'What's new in v0.20' callout with the measured -26%/-32% deltas and a runnable 'Try it' recipe - Line 46 framework list — now includes Hono / Fastify / tRPC / Prisma (was missing all 4) - Line 243 'Honest disclosure' framework list — same correction - Roadmap: added the missed v0.19/v0.20 items (#129 framework-aware boost, #131 value-per-token, #132 signature mode, #133 metadata match, #134 readiness criteria, plus the #130 benchmark receipts). Added #135 (task-conditioned slicing v1) and #84 (Python/Go deeper passes) to Planned. 2) Test recipe — added inline to the 'What's new' block: graphify-ts generate . --spi jq '.nodes[] | select(.framework_role)' graphify-out/graph.json 3) npm video fallback — added a shields.io 'Watch the 30-second demo' button above the bare video URL. npm renders the button; GitHub renders both the button and the inline video. The button links back to the GitHub README anchor where the video plays. 4) Light cleanup only. Larger structural reorg (Why/What/Core concept overlap, section ordering) intentionally NOT in this PR — too risky to bundle with the v0.20 content fixes. Saved for a follow-up. * docs(readme): restructure — 40% shorter, no roadmap, consolidated sections Replaces the previous 20+ section sprawl with 12 focused sections per user feedback ('it looks crowded and messy'). ## New structure (12 sections, down from 20+) 1. Title + pitch + badges 2. Demo (video + npm shields fallback) 3. Quickstart 4. What graphify-ts is (consolidates the old Why + What it does + Core concept + What's it for sections) 5. Measured results (one benchmark table — dropped the ASCII turns walkthrough that was longer than the benchmark itself) 6. Works with your AI tools 7. MCP tools 8. Common commands 9. Trust + limitations (merged old What stays local + Honest disclosure) 10. Documentation & receipts (merged old Public proof + Documentation; CHANGELOG link replaces the in-README Roadmap per user request) 11. Contributors (preserved verbatim with auto-update workflow note) 12. Credit & license ## What was removed (intentionally) - Roadmap section — user explicitly opted out. The CHANGELOG link in Documentation covers this. - 'See it work' ASCII turns walkthrough — verbose and redundant with the benchmark table. - 'Context-plane surfaces' section — its content overlapped with MCP tools + Common commands; not a distinct concept. - Per-use-case prose ('Our AI bill', 'Code review takes hours', 'Can't ship to cloud') — collapsed into 3 bullets in What graphify-ts is. The product positioning is now a few sentences, not 3 sub-headings. ## What was preserved - All benchmark numbers + receipts links - Full agent install table - Full MCP core/full tool surface - Common commands reference - Contributors section with auto-update marker - Credit + license + Safi Shamsi attribution * docs(readme): restore phrases pinned by existing tests (no behavioral change) After the v0.20 README restructure, 6 README-pinning tests failed. Restored the exact phrases each test expects: - 'context plane' + 'context compiler' in the lede (used by package-metadata.test.ts and why-graphify-doc.test.ts to assert product positioning) - '## License' as its own H2 (split from 'Credit & license' into 'Credit' + 'License') - '| **Latency**' bold cell formatting in the benchmark table - 'These six MCP tools' + 'The full surface is 25 tools' phrasing - Full enumeration of full-profile MCP tools (the previous shortening to '...and more' broke the test that pins specific tool names like 'get_neighbors') No structural / length impact — the README is still 12 sections and ~219 lines. These are wording-level restorations to satisfy semantic-content tests. 1833/1833 pass.
* docs(readme): v0.20 refresh — framework substrate audit + npm video fallback + 'what's new' callout + test recipe Honest answers to all 4 user concerns: 1) README was stale for v0.18-v0.20. Updated: - Top-of-file 'What's new in v0.20' callout with the measured -26%/-32% deltas and a runnable 'Try it' recipe - Line 46 framework list — now includes Hono / Fastify / tRPC / Prisma (was missing all 4) - Line 243 'Honest disclosure' framework list — same correction - Roadmap: added the missed v0.19/v0.20 items (#129 framework-aware boost, #131 value-per-token, #132 signature mode, #133 metadata match, #134 readiness criteria, plus the #130 benchmark receipts). Added #135 (task-conditioned slicing v1) and #84 (Python/Go deeper passes) to Planned. 2) Test recipe — added inline to the 'What's new' block: graphify-ts generate . --spi jq '.nodes[] | select(.framework_role)' graphify-out/graph.json 3) npm video fallback — added a shields.io 'Watch the 30-second demo' button above the bare video URL. npm renders the button; GitHub renders both the button and the inline video. The button links back to the GitHub README anchor where the video plays. 4) Light cleanup only. Larger structural reorg (Why/What/Core concept overlap, section ordering) intentionally NOT in this PR — too risky to bundle with the v0.20 content fixes. Saved for a follow-up. * docs(readme): restructure — 40% shorter, no roadmap, consolidated sections Replaces the previous 20+ section sprawl with 12 focused sections per user feedback ('it looks crowded and messy'). ## New structure (12 sections, down from 20+) 1. Title + pitch + badges 2. Demo (video + npm shields fallback) 3. Quickstart 4. What graphify-ts is (consolidates the old Why + What it does + Core concept + What's it for sections) 5. Measured results (one benchmark table — dropped the ASCII turns walkthrough that was longer than the benchmark itself) 6. Works with your AI tools 7. MCP tools 8. Common commands 9. Trust + limitations (merged old What stays local + Honest disclosure) 10. Documentation & receipts (merged old Public proof + Documentation; CHANGELOG link replaces the in-README Roadmap per user request) 11. Contributors (preserved verbatim with auto-update workflow note) 12. Credit & license ## What was removed (intentionally) - Roadmap section — user explicitly opted out. The CHANGELOG link in Documentation covers this. - 'See it work' ASCII turns walkthrough — verbose and redundant with the benchmark table. - 'Context-plane surfaces' section — its content overlapped with MCP tools + Common commands; not a distinct concept. - Per-use-case prose ('Our AI bill', 'Code review takes hours', 'Can't ship to cloud') — collapsed into 3 bullets in What graphify-ts is. The product positioning is now a few sentences, not 3 sub-headings. ## What was preserved - All benchmark numbers + receipts links - Full agent install table - Full MCP core/full tool surface - Common commands reference - Contributors section with auto-update marker - Credit + license + Safi Shamsi attribution * docs(readme): restore phrases pinned by existing tests (no behavioral change) After the v0.20 README restructure, 6 README-pinning tests failed. Restored the exact phrases each test expects: - 'context plane' + 'context compiler' in the lede (used by package-metadata.test.ts and why-graphify-doc.test.ts to assert product positioning) - '## License' as its own H2 (split from 'Credit & license' into 'Credit' + 'License') - '| **Latency**' bold cell formatting in the benchmark table - 'These six MCP tools' + 'The full surface is 25 tools' phrasing - Full enumeration of full-profile MCP tools (the previous shortening to '...and more' broke the test that pins specific tool names like 'get_neighbors') No structural / length impact — the README is still 12 sections and ~219 lines. These are wording-level restorations to satisfy semantic-content tests. 1833/1833 pass. * Implement context compiler payoff Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix PR comments and CI Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix remaining review comments Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Summary
Closes the gap between "SPI substrate exists" and "users see token savings." Two complementary pieces.
1. Confirmed: framework metadata flows end-to-end with
--spiMy earlier claim that the graph.json serializer was dropping
framework_role/route_pathwas wrong — it was a bad test assertion (whitespace formatting mismatch). Tightened the v0.18 test to assert what's actually produced:{ "label": "listUsers()", "framework": "express", "framework_role": "express_route", "node_kind": "route", "route_path": "/users", ... }The serializer in
src/pipeline/export.tsalready spreads all node attributes via...attributes. No serializer change needed.2. Retrieval boost for the v0.17 substrates
frameworkBoostForNode()insrc/runtime/retrieve.tspreviously only handledexpress/redux/reactRouter/nest/next. v0.17 added 4 more substrates (Hono / Fastify / tRPC / Prisma) but retrieval ignored theirframework_rolevalues.Now wired:
hono/fastify/trpc/prismapluginIntent/procedureIntent/queryIntent/mutationIntent/subscriptionIntent/modelIntentframework_rolevalues (hono_route, hono_middleware, hono_app, fastify_route, fastify_plugin, fastify_app, trpc_procedure_query/mutation/subscription, trpc_router, prisma_client)activeFrameworksForProfilenow registershono/fastify/trpc/prismaso framework-compatible filtering recognises themToken / benchmark impact
Framework-shaped questions ("show me hono routes", "what tRPC mutations exist", "find the Prisma client") now favour the structurally-correct nodes via
framework_roleboost. Concretely:context_packresponse surfaces the right nodes inside budget instead of bumping itTest plan
npm run typecheckcleannpm run buildcleannpm run test:run— 1817/1817 pass (110 files)Summary by CodeRabbit
New Features
Tests