Skip to content

v0.3.3

Latest

Choose a tag to compare

@github-actions github-actions released this 23 Oct 14:36

Fixes false LLM idle timeouts while upstream data is still arriving, and separates transport failures from actual timeouts.

  • Keep streaming and non-streaming responses alive on incoming body bytes, including partial NDJSON lines and heartbeat fragments. There is no new total-duration limit; the defaults remain 30s and 90s.
  • Pause the upstream idle timer while processing events or writing downstream. Reuse one timer per request, guard stale callbacks, and stop it on completion without background read buffers or reader goroutines.
  • Return 504 upstream_idle_timeout for the local watchdog, 504 upstream_timeout for other network timeouts, and 502 upstream_connection_error for interrupted connections. Once SSE starts, send the structured error followed by one [DONE] instead of a successful finish.
  • Preserve cancellation and failure accounting. Local I/O failures do not rotate keys, open breakers, or replay requests. Remove the misleading 429 classification and context-reduction hint for network failures.

Regression coverage includes long fragmented events, heartbeats, continuous reasoning, zero-byte reads, idle response headers/body/tail, connection errors, slow downstream writes, client cancellation, stale timer callbacks, and real local HTTP cancellation with key-pool checks. Go race tests, vet and lint pass.

Existing abort, truncated-response and incomplete-tool validation stays in place. Timeout settings in external gateways such as NewAPI remain independent.

Full changelog: v0.3.2...v0.3.3