Skip to content

v0.1.8 — Configurable embedding concurrency

Choose a tag to compare

@AnnaSuSu AnnaSuSu released this 07 Oct 12:43
· 15 commits to main since this release

Indexing can now overlap embedding requests to remote model services with OCE_EMBEDDING_CONCURRENCY. The default remains 1; supported values are 1–8 per workspace. Start with 2 when the endpoint supports concurrent requests.

OCE_EMBEDDING_BATCH_SIZE=64
OCE_EMBEDDING_CONCURRENCY=2

The client bounds in-flight batches, matches out-of-order responses to their source documents, and commits successful batches to the vector cache. If a request fails, it stops submitting new batches and preserves successful in-flight results for retry. Changing concurrency or batch size does not invalidate cached vectors. Server-side GPU inference batch sizes are unchanged.

Save the setting in the shared user configuration (or through setup with environment variables), then restart the MCP client after active indexing finishes.

Validation: 70 Node tests passed on macOS (2 Windows-only tests skipped), 121 Python tests passed (1 platform-specific test skipped), and an isolated install passed setup, doctor, and a real-model MCP indexing/search smoke test with concurrency 2 and 2560-dimensional embeddings. Release artifacts include the npm archive and its SHA-256 checksum.