perf(server): cache HTTP query embeddings - #2192
Merged
RaghavChamadiya merged 1 commit intoSep 11, 2026
Merged
Conversation
justonemorenight
requested review from
RaghavChamadiya and
swati510
as code owners
September 10, 2026 13:36
RaghavChamadiya
approved these changes
Sep 11, 2026
RaghavChamadiya
left a comment
Member
There was a problem hiding this comment.
Leaving KeylessEmbedder bare is the detail that makes this correct, and the reason is worth the two lines you gave it: is_semantic_embedder fails open by design (base.py:185), so a wrapped keyless would come back True and switch on a vector leg with no signal. Checking the raw embedder before wrapping keeps both answers right. One thing to keep an eye on rather than change here: embedder_factory is called once per workspace member (registry.py:242), so a large workspace now holds one 256-entry LRU per repo instead of one. Bounded and only filled by real queries, so not a blocker, but it is worth remembering if a workspace ever reports memory growth on the serve path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Install the existing
CachingEmbedderon the long-lived HTTP query path that #2189 explicitly left as a follow-up.The wrapper is applied both to the primary HTTP vector store and to lazy workspace repo contexts. This keeps one cache per embedder instance/configuration and avoids repeated provider calls for repeated searches.
Safety
KeylessEmbedderbare sois_semantic_embedder()continues to disable the signal-free vector leg._build_embedder()returning the concrete backend for callers that need backend-specific attributes.Tests
uv run ruff check packages/server/src/repowise/server/app.py tests/unit/server/test_app_embedder_backends.pyuv run pytest tests/unit/server/test_app_embedder_backends.py tests/unit/test_persistence/test_caching_embedder.py -q(14 passed)uv run pytest tests/unit/server/test_app_workspace_registry.py tests/unit/server/test_jobs.py -q(10 passed)uv run pytest tests/unit/server --ignore=tests/unit/server/mcp -q(1323 passed)