Skip to content

v0.5.5

Latest

Choose a tag to compare

@npow npow released this 04 Mar 16:02
· 4 commits to main since this release

Fix semaphore slot leak on uncaught exceptions in non-streaming handlers

Previously, if _read_cli_result() raised any exception other than asyncio.TimeoutError, the semaphore slot was never released. After 10 such exceptions the relay would permanently reject all requests with 503 overloaded_error.

Adds except Exception: cleanup + release + raise to all three non-streaming endpoints (/v1/chat/completions, /v1/responses, /v1/messages).