0.69.22

Highlights

Streaming: HTTP connections are now reused across streamed turns

_stream_response_async stopped reading a streamed chat completion at the usage
chunk — before the SSE [DONE] event and before the end of the chunked body. The
SDK stream was therefore abandoned mid-iteration and its response only closed on
garbage collection, which on HTTP/1.1 drops the underlying connection.

The practical cost, measured in production on AudioCodes LiveHub (real-time voice
agents, where the first token is what the caller hears): every streamed turn paid
a fresh TCP + TLS handshake, roughly 600 ms, and any keep-alive pool configured
through http_client_config never took effect. On openai 3.x the abandoned stream
also logged RuntimeError: aclose(): asynchronous generator is already running.

The fix holds a single iterator over the stream and drains whatever the processing
loop leaves behind, so openai.AsyncStream finishes the body itself and the
connection returns to the pool. The loop still processes exactly the events it did
before, so no result changes. Two guards keep the drain off the critical path:

• bounded by a 5s STREAM_DRAIN_TIMEOUT, so an endpoint that keeps a stream open
  after its usage chunk (SSE heartbeat comments) can never block the call — on
  timeout the behavior is exactly the pre-fix state;
• skipped when more than one choice was requested, since with n > 1 the loop
  breaks while the server is still streaming choices the caller discarded.

Connection reuse requires openai >= 3.30; the fix is correct and inert on
openai 2.x. The sync path is deliberately untouched — openai.Stream has no
equivalent drain call, so sync reuse needs an upstream SDK change.

Reported, diagnosed, and fixed by @alexagr (AudioCodes) in #1289; this release
follows their proposed fix, adding only the scope narrowing, the drain timeout,
and the n guard. (#1290)

Dependencies

uv.lock refreshed to clear two Dependabot alerts, each pinned to the minimum
fixed version: oauthlib 4.0.0 → 4.0.1 (high) and filelock 3.32.7 → 4.0.9
(medium, a major bump, transitive only). No pyproject.toml range was touched and
no alert was dismissed. (#1288)

Pull requests in this release

• #1290 — fix(llm): drain streamed async completions so HTTP connections are reused
• #1288 — chore(deps): refresh uv.lock to clear the oauthlib and filelock alerts

Thanks

Thanks to @alexagr for the report, the diagnosis, and the fix in #1289.

Full changelog: 0.69.21...0.69.22