Skip to content

[Feat] Add keywords merge weights and activate the top_k_files for FAST mode#139

Merged
wangxingjun778 merged 2 commits intomainfrom
fix/search
Apr 3, 2026
Merged

[Feat] Add keywords merge weights and activate the top_k_files for FAST mode#139
wangxingjun778 merged 2 commits intomainfrom
fix/search

Conversation

@wangxingjun778
Copy link
Copy Markdown
Member

@wangxingjun778 wangxingjun778 commented Apr 3, 2026

Features

  • Multi-file Evidence Aggregation: Transitioned the fast search mechanism from single-file to multi-file evidence aggregation for more comprehensive context retrieval.
  • IDF-weighted Log-TF Scoring: Implemented a new ranking system using IDF-weighted log-TF scoring to improve result relevance.
  • LLM Keyword Weighting: Updated the LLM prompt to provide keyword importance weights, enhancing the precision of the search logic.
  • Dynamic Context & Fallbacks: Added support for dynamic context windows and introduced file-head fallbacks to ensure better evidence extraction when initial matches are insufficient.

Fixes

  • IDF Scale Consistency: Corrected an inconsistency between the fallback IDF scale and the LLM-defined scale to ensure uniform scoring logic.
  • Filename Search Pagination: Ensured that the filename search fallback correctly respects the top_k parameter, preventing unbounded result sets.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request enhances the fast search mechanism by transitioning from single-file to multi-file evidence aggregation and implementing an IDF-weighted log-TF scoring system for ranking. The LLM prompt is updated to provide keyword importance weights, and the search logic now supports dynamic context windows and file-head fallbacks for better evidence extraction. Feedback focuses on correcting an inconsistency between the fallback IDF scale and the LLM-defined scale, as well as ensuring the filename search fallback respects the new top_k parameter.

Comment thread src/sirchmunk/search.py
Comment thread src/sirchmunk/search.py
Comment thread src/sirchmunk/search.py
@wangxingjun778 wangxingjun778 merged commit b4bd258 into main Apr 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant