Skip to content

feat: KG-boosted file search with ExternalScorer trait (#222)#754

Merged
AlexMikhalev merged 8 commits into
mainfrom
feat/fff-kg-boosted-file-search
Apr 3, 2026
Merged

feat: KG-boosted file search with ExternalScorer trait (#222)#754
AlexMikhalev merged 8 commits into
mainfrom
feat/fff-kg-boosted-file-search

Conversation

@AlexMikhalev

Copy link
Copy Markdown
Contributor

Summary

Integrates FFF (Fast File Finder) scoring engine with terraphim_automata Aho-Corasick knowledge graph matching to create domain-aware file search for AI agents.

  • New terraphim_file_search crate with KgPathScorer that boosts files containing KG domain terms
  • terraphim_find_files MCP tool: fuzzy file search with KG-boosted ranking
  • terraphim_grep MCP tool: content search with KG-aware file ordering
  • KgWatcher: hot-reload thesaurus on directory changes (notify + 500ms debounce)
  • Criterion benchmarks for scoring performance

Key design decisions

  • ExternalScorer trait in fff-core fork (~25 line diff) enables pluggable scoring
  • KgPathScorer uses Aho-Corasick on file.relative_path (under 2ms for 10K files)
  • Scoring formula: min(unique_kg_terms * 5, 30) -- conservative, configurable
  • RwLock for thread-safe hot-reload under rayon parallel scoring

Test plan

  • 8 unit tests (kg_scorer + watcher)
  • 5 integration tests (find_files + grep MCP tools)
  • 13 existing MCP server tests still pass
  • Security review: PASS
  • Benchmarks: path scoring under 2ms/10K files

Refs #222 #227

AlexMikhalev and others added 5 commits April 3, 2026 14:52
…Refs #227

Scaffold terraphim_file_search crate integrating fff-search ExternalScorer
with terraphim_automata knowledge-graph matching. KgPathScorer scores files
by counting unique KG term matches in relative_path with configurable
weight_per_term (default 5) and max_boost (default 30). Includes 6 unit
tests covering empty thesaurus, path match, no match, multiple terms, cap
at max_boost, and hot-reload via update_thesaurus().

Co-Authored-By: Terraphim AI <noreply@terraphim.ai>
#227

- Add fff-search and terraphim_file_search deps to terraphim_mcp_server
- Add kg_scorer: Option<Arc<KgPathScorer>> field to McpService
- Add with_kg_scorer() builder method for optional KG scorer injection
- Implement find_files() method: synchronous FilePicker scan + fuzzy
  search + KG post-processing re-rank (adds KgPathScorer.score() to
  fuzzy total, sorts descending)
- Expose terraphim_find_files MCP tool with query/path/limit params
- Register in list_tools and call_tool dispatch
- Add 3 integration tests in tests/test_find_files.rs

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…efs #227

Step 6: terraphim_grep MCP tool
- Add grep_files() to McpService using fff-search grep_search with
  KG-aware file ordering (highest KgPathScorer score searched first)
- Register terraphim_grep in list_tools and call_tool
- Supports query, path, limit, output_mode (content|files) params
- Add 2 integration tests: content mode and files-only mode

Step 7: KgWatcher hot-reload
- Add notify = "8" + notify-debouncer-full = "0.6" to terraphim_file_search
- New crates/terraphim_file_search/src/watcher.rs: KgWatcher watches a
  directory with 500ms debounce and calls update_thesaurus() on change
- load_thesaurus_from_dir() merges all JSON files in the watched dir
- 2 unit tests: watcher_creates_without_error, watcher_triggers_update

All existing tests pass (6 kg_scorer + 3 find_files integration).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Apr 3, 2026

Copy link
Copy Markdown
Contributor

Documentation Preview

Your documentation changes have been deployed to:
https://1ac34ec3.terraphim-docs.pages.dev

This preview will be available until the PR is closed.

Switch fff-search from a local filesystem path (which doesn't exist on
CI runners) to a git dependency on AlexMikhalev/fff.nvim fork with the
ExternalScorer trait. Also fix u64 ID type mismatches from the recent
ID refactor, fix pre-existing llm_router test failures (missing
llm_enabled flag), resolve clippy warnings, and apply cargo fmt.

Refs #222 #227

Co-Authored-By: Terraphim AI <noreply@anthropic.com>
@github-actions

github-actions Bot commented Apr 3, 2026

Copy link
Copy Markdown
Contributor

Documentation Preview

Your documentation changes have been deployed to:
https://af03dc88.terraphim-docs.pages.dev

This preview will be available until the PR is closed.

The fff-search build.rs requires the zlob feature when the CI env var
is set. Add --features zlob to clippy, check, and test commands in
ci-pr.yml. Also fix terraphim_file_search Cargo.toml where dependencies
were misplaced under [features] section. Refs #227

Co-Authored-By: Terraphim AI <noreply@anthropic.com>
@github-actions

github-actions Bot commented Apr 3, 2026

Copy link
Copy Markdown
Contributor

Documentation Preview

Your documentation changes have been deployed to:
https://81060d21.terraphim-docs.pages.dev

This preview will be available until the PR is closed.

@AlexMikhalev

Copy link
Copy Markdown
Contributor Author

retrigger

@github-actions

github-actions Bot commented Apr 3, 2026

Copy link
Copy Markdown
Contributor

Documentation Preview

Your documentation changes have been deployed to:
https://d89fa189.terraphim-docs.pages.dev

This preview will be available until the PR is closed.

Fixes clippy::len_zero warning triggered on CI. Refs #227

Co-Authored-By: Terraphim AI <noreply@anthropic.com>
@github-actions

github-actions Bot commented Apr 3, 2026

Copy link
Copy Markdown
Contributor

Documentation Preview

Your documentation changes have been deployed to:
https://59649abf.terraphim-docs.pages.dev

This preview will be available until the PR is closed.

@AlexMikhalev AlexMikhalev merged commit 28017c3 into main Apr 3, 2026
33 checks passed
@AlexMikhalev AlexMikhalev deleted the feat/fff-kg-boosted-file-search branch April 3, 2026 17:26
AlexMikhalev added a commit that referenced this pull request Apr 25, 2026
feat: KG-boosted file search with ExternalScorer trait (#222)
AlexMikhalev added a commit that referenced this pull request May 1, 2026
feat: KG-boosted file search with ExternalScorer trait (#222)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant