Repository navigation
Partner v0.2.1
Partner v0.2.1
A fix release for one bug in v0.2.0's Memory Services: a streaming request to a service returned the whole answer and then the connection died, so callers saw "terminated" instead of the end of the reply. It only happened with Summaries enabled, which is the only mode where the proxy parses the stream at all. Nothing else changed — the session, memory, settings and workspace formats are untouched, and this is a drop-in replacement for v0.2.0.
The fix
A stream no longer ends in a dropped connection when Summaries is on
When summaries are enabled, the service reads the upstream's SSE frames as it forwards them, to keep the text it will later summarize. It read choices[0] unconditionally.
A stream that ends with a usage frame — what OpenAI sends when the request carries stream_options: { include_usage: true }, and what Azure, vLLM, LiteLLM and several other OpenAI-compatible servers send by default — looks like this:
data: {"choices":[{"delta":{"content":"answer"}}]}
data: {"choices":[],"usage":{"prompt_tokens":812,"completion_tokens":34}}
data: [DONE]
That second frame has an empty choices array. Indexing [0] on it threw ArgumentOutOfRangeException inside the response copy loop, which is past the point where headers have been sent: the error could no longer become an HTTP status, so Kestrel dropped the connection and the caller's HTTP client reported the stream as terminated. The forwarded text itself had already arrived, which is why it looked like a truncation rather than a failure.
The frame is now read through a helper that answers null for an empty array, so the usage frame is simply nothing to capture — exactly like a [DONE] sentinel or a keep-alive comment. The same guard covers the non-streaming body parse, where a provider that answers with choices: [] had the same hole.
Pass-through is unchanged: the usage frame is still forwarded byte-for-byte to the caller. Only the summary capture skips it.
Upgrading
- Install v0.2.1 directly, or let v0.2.0 update itself.
- No data or format changes: the memory database stays at schema v3, and settings, sessions and downloaded engines are untouched.
- If you were running a service with summaries on and streaming from it, this is the release that stops those callers seeing "terminated".
Engineering
ServiceProxyTestsgained a case that streams a delta, then an empty-choicesusage frame, then[DONE], and asserts the captured stream reaches[DONE]and the round is recorded (pendingRounds= 1). It fails against v0.2.0 with the sameArgumentOutOfRangeExceptiona caller saw.
The voice engines are downloaded from the engines-v1 release on first use; their URLs and SHA-256 are pinned in this build. The app code is MIT licensed; the engine runtimes and model weights keep their own licences, see THIRD-PARTY-NOTICES.md in the release assets.