Extract shared knowledge base search and evaluation logic to submodule#85
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Extracts the knowledge-base search utilities (previously in mcp-local/utils/search_utils.py) and retrieval evaluation helpers into a new standalone arm_kb_search Python package, and updates mcp-local/server.py, the Dockerfile, and the evaluation script to consume it.
Changes:
- New
arm_kb_searchpackage withsearch,loaders,evaluation,configmodules and apyproject.toml. mcp-local/server.pyand Dockerfile updated to import fromarm_kb_search;DISTANCE_THRESHOLD/K_RESULTSremoved frommcp-local/utils/config.py.embedding-generation/evaluate_retrieval.pyreplaced byevaluate_local_retrieval.py, which delegates toarm_kb_search.evaluation; shell script and README updated accordingly.
Reviewed changes
Copilot reviewed 13 out of 13 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| pyproject.toml | Declares the new arm-kb-search package and its dependencies. |
| arm_kb_search/init.py | Public API re-exports for the new submodule. |
| arm_kb_search/config.py | Hosts moved DISTANCE_THRESHOLD/K_RESULTS constants. |
| arm_kb_search/loaders.py | Index/metadata loader helpers moved out of search.py. |
| arm_kb_search/search.py | Removes loader helpers and json/os imports. |
| arm_kb_search/evaluation.py | New shared retrieval evaluation helpers with dataclasses. |
| mcp-local/server.py | Switches imports to arm_kb_search. |
| mcp-local/utils/config.py | Removes constants that moved to arm_kb_search.config. |
| mcp-local/Dockerfile | Copies the arm_kb_search package into the image. |
| embedding-generation/evaluate_local_retrieval.py | New evaluation entrypoint built on arm_kb_search. |
| embedding-generation/evaluate_retrieval.py | Removed (replaced by new script). |
| embedding-generation/run-question-eval.sh | Invokes the renamed evaluation script. |
| embedding-generation/README.md | Updates the script name mentioned in docs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
JoeStech
requested changes
May 15, 2026
Collaborator
Author
|
@JoeStech Addressed the comments and also updated the evaluation function to output both the summary and actually include the individual cases so that we can do more detailed breakdowns later. |
JoeStech
approved these changes
May 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.