Skip to content

v3.3.0

Latest

Choose a tag to compare

@vitobotta vitobotta released this 07 Oct 21:12
dd8dced

What's New

  • Per-model ttft_timeout. Each model entry can now set its own ttft_timeout, overriding the global timeouts.ttft for streams to that model — useful when one model's reasoning phase legitimately delays the first content token while others must fail fast. Models without the key keep the global default, and ttft_timeout: false (or blank) disables the TTFT gate for that model alone. Out-of-range values are rejected at boot and reload, same as the global setting.

Code organisation

  • ConfigStore.ttft_timeout_for(name) resolves the effective value (model override, else timeouts.ttft), and try_stream takes model_name: to look it up per attempt. config/config.yaml.example, README and AGENTS.md document the new key.