v2.6.0

What's New

• TTFT timeout for streaming requests — a new timeouts.ttft config key (seconds) lets the proxy fall back to the next provider when the primary takes too long to start generating. When a streaming request doesn't receive a first token (thinking or content) within the configured deadline, the proxy treats it as a failure: it retries on the same provider up to max_attempts, then falls back to the next provider — the same path as any other failure.

  timeouts:
    open: 30
    read: 300
    write: 60
    ttft: 15   # max seconds to wait for the first token (streaming only)

  Set ttft to null or omit it to disable the feature.

  How it works: An application-level check in consume_stream uses a total deadline (request_start + ttft_timeout) to detect when no first token has arrived. The check runs after each parsed chunk — if tracker.first_token is still nil and the deadline has passed, TTFTTimeoutError is raised before the chunk is forwarded to the client.

  Stale pooled connections are not affected — they continue to be caught by the existing EOFError retry path (<1s, free, doesn't consume the attempt budget). The normal read_timeout (300s) handles truly dead connections.

  The TTFTTimeoutError bypasses the mid-stream corruption guard (which normally prevents retries after data has been forwarded) — safe because no content was sent to the client, only comment lines that SSE clients ignore.

  Scope: streaming only. Non-streaming requests and background probes are unaffected. Requires tracking.enabled: true (the default) — chunk parsing is needed to detect the first token.

Performance

• HTTPSupport::TTFTTimeoutError — new exception class, rescued with a distinct "TTFT timeout" label in logs and "ttft_timeout" in Prometheus metrics.

Configuration

• timeouts.ttft — new optional key. Validated as a positive number (1..86400 seconds). When omitted, the feature is disabled and behaviour is unchanged.

Tests

• 15 new tests: 6 for consume_stream TTFT behaviour, 4 for try_stream (timeout detection, content within deadline, ping within deadline, tracking-disabled skip), 5 for config validation.
• 59 runs, 130 assertions, 0 failures.