Skip to content

Release Notes - v0.4.0

Choose a tag to compare

@zhixiangxue zhixiangxue released this 02 Jun 09:26
· 4 commits to main since this release

Resilient Provider Failover

Configure fallbacks to automatically fail over to backup LLM providers when the primary is unavailable.

conv = chak.Conversation(
    "anthropic/claude-haiku-4-5",
    api_key="...",
    fallbacks=[
        {"model_uri": "openai/gpt-4o-mini", "api_key": "..."},
        {"model_uri": "deepseek/deepseek-v4-flash", "api_key": "..."},
    ],
)

Behavior:

  • Errors like timeout, connection failure, rate-limit (429), and server errors (5xx) trigger automatic failover to the next provider in order.
  • Client errors (401, 400, 404) do not trigger failover — they raise immediately.
  • Works for both streaming and non-streaming calls.
  • response.metadata.provider_trace tells you exactly what happened: which provider succeeded, how many failed, and why.

Improvements

  • Structured error types: All provider errors are now classified into canonical types (timeout, connection_error, rate_limit, server_error, auth_error, bad_request, not_found) for precise failover decisions.
  • ErrorType.from_status_code(): Deterministic HTTP status → error type mapping, available as a classmethod.
  • Provider trace metadata: Every response (resilient or not) carries provider_trace with primary/resolved provider info and failure history.