Skip to content

perf(server): cache HTTP query embeddings - #2192

Merged
RaghavChamadiya merged 1 commit into
repowise-dev:mainfrom
justonemorenight:codex/cache-http-query-embeddings
Sep 11, 2026
Merged

perf(server): cache HTTP query embeddings#2192
RaghavChamadiya merged 1 commit into
repowise-dev:mainfrom
justonemorenight:codex/cache-http-query-embeddings

Conversation

@justonemorenight

Copy link
Copy Markdown
Contributor

What

Install the existing CachingEmbedder on the long-lived HTTP query path that #2189 explicitly left as a follow-up.

The wrapper is applied both to the primary HTTP vector store and to lazy workspace repo contexts. This keeps one cache per embedder instance/configuration and avoids repeated provider calls for repeated searches.

Safety

  • Leaves KeylessEmbedder bare so is_semantic_embedder() continues to disable the signal-free vector leg.
  • Reuses the bounded, copy-on-read LRU from perf(mcp): cache query embeddings so a repeated search skips the provider #2189; no new cache policy or persistence.
  • Keeps _build_embedder() returning the concrete backend for callers that need backend-specific attributes.
  • Bulk embedding remains pass-through, so indexing batches do not fill the query cache.

Tests

  • Repeated HTTP query text reaches the underlying embedder once.
  • Keyless HTTP embedder is not wrapped.
  • uv run ruff check packages/server/src/repowise/server/app.py tests/unit/server/test_app_embedder_backends.py
  • uv run pytest tests/unit/server/test_app_embedder_backends.py tests/unit/test_persistence/test_caching_embedder.py -q (14 passed)
  • uv run pytest tests/unit/server/test_app_workspace_registry.py tests/unit/server/test_jobs.py -q (10 passed)
  • uv run pytest tests/unit/server --ignore=tests/unit/server/mcp -q (1323 passed)

@RaghavChamadiya RaghavChamadiya left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leaving KeylessEmbedder bare is the detail that makes this correct, and the reason is worth the two lines you gave it: is_semantic_embedder fails open by design (base.py:185), so a wrapped keyless would come back True and switch on a vector leg with no signal. Checking the raw embedder before wrapping keeps both answers right. One thing to keep an eye on rather than change here: embedder_factory is called once per workspace member (registry.py:242), so a large workspace now holds one 256-entry LRU per repo instead of one. Bounded and only filled by real queries, so not a blocker, but it is worth remembering if a workspace ever reports memory growth on the serve path.

@RaghavChamadiya
RaghavChamadiya merged commit 3c93e0f into repowise-dev:main Sep 11, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants