Skip to content

backend-v0.9.3

Choose a tag to compare

View all tags
@kwoo24-oss kwoo24-oss released this 24 Jun 07:42
· 525 commits to main since this release
Immutable release. Only release title and notes can be modified.
backend-v0.9.3
a286c90
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

fix(search): tokenize on a ProcessPool, not to_thread (event-loop starvation → 503) (#244)

Kiwi (BM25 sparse tokenizer) is a CPU-bound C++ extension that does not release the GIL during native tokenize(). Running it via asyncio.to_thread therefore serialized concurrent tokenizations on the GIL and starved the single event-loop thread → /livez couldn't be scheduled → readiness/liveness probes timed out → pod pulled from Service → intermittent 503 flapping under search + indexing load.

sparse_encoder.tokenize() now offloads to a dedicated spawn ProcessPoolExecutor (each worker its own GIL; initializer warms Kiwi per worker), created/torn-down in the lifespan; falls back to to_thread when the pool isn't started (tests). search/embed_worker/stats-refresher all benefit. Search results unchanged; RWO/single-replica unaffected.

Measured locally: event-loop max stall ~472 ms (to_thread, fully serial) → ~13–34 ms (ProcessPool), ~4× tokenization throughput.

Assets 3
Loading