Skip to content

v0.3.4 — routing-cell failover

Latest

Choose a tag to compare

@yfwu2020 yfwu2020 released this 06 Nov 08:45

Why

The connection loaded fine, but long conversations could not send a single token. The OpenCode Zen Go gateway pins every request carrying the same x-opencode-session to one backend cell, and a cell short on prefill capacity answers 429 [prefill_cap_refused] in about a second. Retrying the same session id kept hitting the same cell, so the whole conversation failed while a new one worked.

Measured against the live gateway with one identical body:

Attempt Body Result
stuck session id ×3 250k chars 429 prefill_cap_refused in 1.2–2.2s, 3/3
three random ids 250k chars served 2/3 (one transient overloaded), 11–16s
fresh id 700k chars served in 62s
stuck session id 700k chars refused in 3.7s

What changed

One refusal now advances that conversation to <sessionId>#r<n>; the host's existing retry then lands on another cell, and later requests keep using the derived id so the new cell keeps its prompt cache.

  • healthy conversations keep their original id and cache affinity;
  • transient [overloaded] answers do not rotate;
  • rotations are bounded at 32 per session and tracked for at most 256 sessions;
  • README documents the behaviour and the per-provider retryPolicy knob for longer backoff.

Validation

27 tests pass, including three new ones: a refusal rotates once, a healthy stream never rotates, and an always-refusing endpoint stops at #r32. npm ci accepts the lockfile and the release tag agrees with 0.3.4.