Skip to content

v2.13.2

Choose a tag to compare

@andrew-hernandez-paragon andrew-hernandez-paragon released this 11 Mar 22:33
· 94 commits to main since this release

Fix: Concurrency semaphore prevents embedding timeouts

When parallel workers (15+) send large embed requests simultaneously, the sidecar's GPU processes them serially — later requests queue in uvicorn and timeout at 60s.

Fix: A semaphore in EmbeddingSidecar.embed() limits concurrent requests to 2 (configurable via EMBEDDING_MAX_CONCURRENT). Excess callers queue in Node.js with no timeout pressure. The 60s timer only starts after acquiring a slot.

Before: 15 workers × 50 large texts → 15/15 timeout at 60s
After: 15 workers × 50 large texts → 15/15 succeed

Combined with v2.13.1's HTTP batch cap, this completes the fix for large codebase parsing failures introduced in v2.13.0.