Routing and Fallback

Routing and Fallback

A Route is a named routing policy that clients address like a model. It holds an ordered list of targets (an account + upstream model, optionally with a typed predicate and parameter overrides), a selection strategy, fallback triggers, a state-portability policy, and optional prompt-cache affinity.

Resolving a requested name

When a client asks for a model name, Kinetix resolves it in this order:

1. An alias with that name (exact match).
2. A Route with that name.
3. provider/model-id (provider matched by name or id).
4. A bare upstream model id.

Aliases point at either a single model (target_type = "model") or a Route (target_type = "route").

Targets

Each Route target is an (account, model) pair. If a target omits the account, Kinetix uses the lowest-priority non-disabled account of the model's provider. Targets may span providers and wire formats (FR-12.2/12.5/12.10).

Selection strategies

Strategy	Behavior
priority	Try targets in priority order.
round-robin	Rotate the starting target per request.
weighted	Choose proportionally to target weight.
least-used	Prefer the target with the fewest lifetime requests.

Predicates (FR-12.3/12.4)

A target may carry a typed, side-effect-free predicate — an expression tree with three-valued evaluation (true / false / unknown) and a human-readable explanation. Arbitrary code/eval is forbidden. When a fact is unknown, the target's when_unknown policy decides (skip by default, or allow).

Fact vocabulary:

Fact	Meaning
has_tools	The request carries tool definitions.
has_images	The request carries image parts.
has_reasoning	The request sets a thinking/reasoning control.
frontend	openai or anthropic.
requested_model / requested_alias / requested_route	Names the client used.
key_tag	The virtual key's tag.
input_tokens	Known input size.
target_model / target_model_id	The candidate model.
target_provider / target_provider_id	The candidate provider.
target_capability(arg)	A declared capability (unknown if unconfigured).
target_context_window / target_max_output_tokens	Model limits.

Wire format (as stored/returned by the admin API):

{
  "expr": { "fact": "has_tools", "op": "eq", "value": true },
  "when_unknown": "skip"
}

op is one of eq, ne, lt, lte, gt, gte, in, not_in, contains (default eq). fact is a bare name string or { "name": ..., "arg": ... }.

Example: send tool requests to a tool-capable model, everything else to a cheap one:

targets: [
  { "model": "Gemini/gemini-2.5-flash", "predicate": { "expr": { "fact": "has_tools", "op": "eq", "value": true } } },
  { "model": "9router/free",            "predicate": { "expr": { "fact": "has_tools", "op": "eq", "value": false } } }
]

Fallback and the commit point (FR-4.5)

• Retry/fallback is allowed only before the first client byte (the commit point). The state machine is SELECT → CONNECT → WAITING_FOR_FIRST_BYTE → COMMITTED → STREAMING/COMPLETE.
• After commit, a failure terminates the stream with a format-correct error (OpenAI: an error object + a chunk with finish_reason: "error" + [DONE]; Anthropic: a single terminal error event). Nothing is silently spliced.
• The attempt loop is bounded by max_attempts (default 5, capped) and a 30-second pre-commit deadline, with bounded backoff between attempts (100 ms → 1 s).
• Key-level failures update account state: 429 → cooldown (honoring Retry-After, short hint = rate limit, long/absent = quota), quota → exhausted, auth → disabled, 5xx/connection/timeout → short cooldown. A per-account circuit breaker opens after repeated failures and probes half-open.

Portability policy (FR-2.11)

When a fallback crosses providers, opaque provider state (e.g. reasoning signatures) cannot travel. The Route's portability_policy decides:

• strip_with_warning (default): remove the non-portable state, record it in the Route Trace, and emit an X-Kinetix-Warning response header. Silent stripping is forbidden.
• reject: fail the request with a format-correct error rather than stripping.

Prompt-cache affinity (FR-7.3/7.5)

When a Route has cache_affinity and the request carries an explicit session header, Kinetix remembers the last successful target for that session and prefers it if still eligible, improving upstream prompt-cache hits. Session identity is taken only from an explicit header — never guessed:

X-Kinetix-Session | X-Session-Id | X-Conversation-Id | X-Session-Affinity | Session-Id

Client-visible routing headers (FR-12.15)

Clients see only opaque routing metadata:

Header	Meaning
X-Request-Id	The request id.
X-Kinetix-Route-Id	Opaque krt_… id; resolvable to a Route Trace by an admin.
X-Kinetix-Cache	hit / miss / bypass.
X-Kinetix-Fallback	1 when a fallback occurred (omitted otherwise).
X-Kinetix-Warning	JSON array of warnings (e.g. a portability strip).

Serving account/provider names are not exposed to clients. An admin can resolve the opaque id at GET /admin/api/route-traces/{opaque_id}.

Related

• Admin API — create/manage Routes and run a Route Dry Run.
• Observability — the Route Trace and flight recorder.