Phase 9: Agent search tooling (context-grep extension)#24
Merged
Conversation
- Convert all source files (parse, ast, enrich, navigate, index) to .ts - Convert test files to .ts, run via tsx - Remove manual .d.mts type declarations (inferred from source now) - Add .mjs and .cjs to LANGUAGE_RULES (same kinds as .js) - Remove dead FOCUS_SIGNALS constant from navigate module - Add tsconfig.json for tools/pi/context-grep (strict, nodenext) - Update shim tsconfig to include source files for joint typecheck - Add @types/node and tsx as root devDependencies - Simplify eslint/knip ignores to tools/pi/** - Update both READMEs with current file layout and supported languages - Add typecheck:pi-tooling script All gates pass: pnpm check, test:pi-tooling (48/48), typecheck:pi-shim, typecheck:pi-tooling.
Rearchitect based on experiment evidence (codeindex-exploration ANALYSIS.md):
the proven mechanism for reducing turns is AST containers + back-references
in a single output block, NOT navigation maps that encourage more exploration.
Key changes:
- Remove navigate.ts (navigation map, lanes, task focus, suggested reads)
- Add backrefs.ts (caller lookup via rg --json --fixed-strings)
- Simplify enrich.ts to single-block output (original + AST context)
- Back-references shown for definitions (grep hit lands on startLine)
- Callers found via rg, filtered (skip imports, tests, re-definitions)
- ast.ts: expose findEnclosing/extractName, sort containers smallest-first
- Keep all safety constraints (heredoc/script rejection, command detection)
Output format now matches what reduced turns by 33% in controlled experiment:
▶ file:start-end [kind name] (grep hits: [...])
│
│ Called from:
│ ← callerName (file:line)
│
function body...
What was removed and why:
- Navigation map suggested more files to read → encouraged divergence
- Lane classification / task focus → no proven value
- Multi-block content return → single block is what the model processes
- Suggested reads → creates a to-do list that increases turns
- Rewrite phase doc to reflect shipped architecture (back-refs, no nav map) - Add experimental validation section with A/B results - Document what was built and removed, with reasons - Add conclusion: extension as passive guardrail for call-chain awareness - Update both READMEs to match current implementation and measured impact - Mark phase status: complete
CI doesn't have ast-grep installed. Tests that require it (container extraction, enrichment integration, back-references) now check for ast-grep availability at module level and skip gracefully. Parser tests, command detection tests, and hardening fixtures run unconditionally.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Project-local Pi extension that enriches grep/rg search results with AST context and back-references. The agent searches as usual; enrichment is appended transparently.
Architecture
Based on the proven codeindex-exploration experiment (n=6, controlled A/B):
Measured impact
The extension acts as a passive guardrail: agents see call chains automatically with every search, reducing uninformed edits.
What's included
tools/pi/context-grep/src/— TypeScript source (parse, ast, backrefs, enrich).pi/extensions/context-grep/— thin Pi shimWhat was tried and removed
A navigation-map layer (lanes, task focus, suggested reads) was implemented and then removed after A/B testing showed it increased turns by encouraging divergence rather than convergence.
No product code changes
This is developer/agent tooling only. No changes to bproxy protocol, CLI, daemon, or browser extension.