Repository navigation
Per-(provider, model) unavailability marking, Retry-After aware
Built live against a real production incident: coding-premium's muse-spark-1.3 tier was overloaded upstream (503 service_overloaded/500/504, intermittently hanging up to 90s). A real curl against the live upstream confirmed it sends a standard Retry-After: 60 header on its 503 — this gateway read the error body but never the response headers at all, so that signal was silently discarded.
Operator's own framing, which drove the design: the request is the unit, not the provider. If a target fails, mark it unavailable; the next real request is itself the retry (honoring the vendor's own Retry-After, or a configurable default backoff otherwise) — if it fails too, it falls through to the next tier exactly like any other failed attempt, invisibly to whoever's waiting on it. No separate background prober.
New internal/unavail package
A simpler, always-on sibling to the existing per-(alias,tier) circuit breaker:
- Keyed by (provider, upstream_model) directly — a target's health is a fact about the vendor/model, not about any one alias's fallback chain, so a failure discovered via one alias immediately protects every other alias routing to the same target.
- No error-rate window, no consecutive-failure counter, no half-open probe-admission race.
- Honors the upstream's own
Retry-Afterexactly when present (seconds or HTTP-date, RFC 9110 §10.2.3); absent that, a configurable default backoff doubles from the target's last mark, capped (built-in default 200ms → ~25.6s over 8 doublings). - Redis-shared across replicas, fails open if Redis is unreachable.
Also
providers.Error.RetryAfter, threaded through every non-2xx upstream call site (chat, audio, model-catalog listing).- Vendor
X-RateLimit-*headers logged when present (observability only, no behavior tied to them yet). - Wired into
executePlanalongside the existing breaker, reusingcountsAgainstTier's classification so both mechanisms agree on what's attributable to a target.
Test plan
internal/unavail: 7 unit tests,-raceclean. Two genuine bugs caught and fixed by these tests before reachingexecutePlan: Retry-After wrongly clamped by the default max, and a unit-scaling bug that silently reset the backoff every other mark.internal/providers: Retry-After parsing (including the exact60confirmed live), httpError population, rate-limit-header logging.internal/httpapi: 3executePlanwiring tests (skip-then-retry, Retry-After-over-default, cross-alias sharing) — surgical-revert proven.- Full suite green with
-race.
Shipped via #161, chart bump #162.
Full Changelog: v0.1.77...v0.1.78