feat: v0.4 search improvements — ranking, round-trip reduction, token efficiency, infrastructure#9
Merged
Merged
Conversation
Three changes that directly address 13/13 benchmark failures (all search misses): - def_weight: semantic scoring per AST node kind (100 for fn/struct/trait, 90 for impl, 80 for const, 30 for export) replaces flat +1000 bonus - basename_boost: file stem matching (exact +300, prefix +150, substr +100) so Walk→walk.rs ranks first - impl collector: piggybacks on definition walker to detect trait impls and interface implementations, surfaced with [impl] label Also: split early quit thresholds (50 defs / 30 usages), trait_item in outline collection. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…eted results Three features that reduce search round-trips: - Siblings: expanded methods show `── siblings ──` footer with referenced fields/methods from the same struct/class (tree-sitter queries for Rust/Python/TS/JS/Go/Java, cap at 6) - Transitive callees: `── calls ──` footer now shows 2-hop callees with tree indentation (→ prefix), BFS with cycle detection, budget=15 for 2nd hop - Faceted results: >5 matches grouped into Definitions, Implementations, Tests, Usages (same/other package) Also: Clone derive on Match for faceting, removed dead code from FacetedResult. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…annel noise Three optimizations reducing token output per expanded match: - strip.rs: cognitive load stripping — removes debug logging, plain comments (preserving TODO/FIXME/doc), collapses consecutive blanks. Per-language rules for Rust/Python/Go/JS/TS/Java/C. - truncate.rs: smart truncation — functions >80 lines compressed to ~40 most diverse lines (control flow, calls, error handling scored highest). Gap markers show omitted line count. - Channel noise: skip outline context for <50-line files, skip leading import blocks in expanded defs, collapse blank line runs. 35 unit tests covering all three modules. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…iring Symbol index (src/index/symbol.rs): - DashMap<Arc<str>, Vec<SymbolLocation>> for O(1) symbol→file lookup - Lazy build on first search via rayon parallel walker - Per-file mtime tracking for incremental invalidation - Reuses existing tree-sitter definition extraction Bloom filters (src/index/bloom.rs): - Per-file probabilistic set membership (1% FPR, ~120 bytes/filter) - Byte-level identifier extraction (no tree-sitter needed) - DashMap cache with mtime invalidation - Pre-filters callee/caller resolution: skip files that definitely don't contain target symbol Wiring (mcp.rs, search/mod.rs, callees.rs, callers.rs): - Index + bloom threaded through MCP server → search → callee/caller - Bloom pre-filter in resolve_callees and find_callers 55 unit tests + 1 doc-test, all passing. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…move premature index build - Extract shared package_root() to search/mod.rs, rank.rs and facets.rs delegate - Hoist detect_file_type() above both truncation and callee sections in format_matches - Remove index.build() from search path — index not yet used for lookup Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
MCP instruction changes: - Tool descriptions: "Replaces X and the host Y tool — use this for all Z" - SERVER_INSTRUCTIONS: explicitly name host tools (Read, Grep, Glob) to replace - EDIT_MODE_INSTRUCTIONS: consistent host tool naming Benchmark infrastructure: - Strip backticks in ground truth checking (markdown formatting) - Relax rg_trait_implementors ground truth (word order) - Add parse.py for result analysis, improve run.py verbosity Results (Sonnet 4.5, 52 runs): - $/correct: $0.26 baseline → $0.19 tilth (-29%) - Accuracy: 96% baseline, 92% tilth - Tool adoption: 98% (185/188 calls used tilth) - W19 T3 L4 Multi-model: - Opus 4.6: 5/5 hard tasks, 100% adoption, $0.23/correct - Haiku 4.5: 18/26 (69%), 42% adoption, $0.19/correct Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Haiku (52 runs): +12pp accuracy (58% → 69%) but +12% $/correct Opus (10 runs): -22% $/correct, 100% accuracy both modes Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Average across 114 runs (3 models): -18% $/correct, 79% → 82% accuracy. 9/26 Haiku tilth runs used zero tilth tools — flagged in README. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Verified against raw JSONL result files. Percentages (98%, 42%) unchanged. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Major search quality and efficiency improvements across 4 tiers, plus instruction tuning for near-total tool adoption.
def_weightscoring by AST node kind,basename_boostfor file-stem matching, impl/interface collector for auto-surfacing trait implementations── siblings ──footer), transitive callee expansion (2-hop BFS with cycle detection), faceted result grouping (definitions/implementations/tests/usages)DashMap+ rayon parallel build), bloom filter cache (per-file probabilistic membership with mtime invalidation), pipeline wiring into search/callees/callersBenchmark results (Sonnet 4.5, 52 runs)
W19 T3 L4 — tilth wins 19 of 26 tasks on cost.
Multi-model: Opus 5/5 hard tasks (100% adoption), Haiku 18/26 (42% adoption).
Review fixes
package_root()across rank.rs and facets.rs into shared mod.rs functiondetect_file_typeabove both uses informat_matches()index.build()call (index not yet used for search fast-path)Stats
Test plan
cargo build && cargo clippy -- -D warnings && cargo test— all cleancargo build --release— optimized build succeeds🤖 Generated with Claude Code