Repository navigation
llm-pi-ai: a gateway's mid-stream interruption wording bypasses retry classification #8743
apple-ouyang
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Summary
classifyPiAiErrorin@deepseek-ai/dsh-llm-pi-aimaps an upstream failurewording to a non-retryable code, so
@deepseek-ai/dsh-llm-retrynever retriesit.
When a gateway in front of the provider reports a mid-response upstream cut as
plain prose with no HTTP status prefix, the message matches none of the
transport rules and falls through to
PI_AI_ERROR:Evidence
From DSH session logs (0.1.7-rc.2), failed turns grouped by failure code and
retry events:
llm/retryeventsSERVER503: {"message":"Service temporarily unavailable",...}SERVER502: {"message":"Upstream service temporarily unavailable",...}PI_AI_ERRORUpstream response stream was interruptedThe
SERVERruns appendllm/retry+llm/retry-startedbeforeturn/end;the
PI_AI_ERRORruns go straight toturn/endwith no retry at all. Both aremid-response connection losses, so they should classify the same way. The
existing
2026-07-22notepi-ai-transport-truncation-classificationalreadysets that principle.
Proposed fix
Classify the wording as
TRANSPORT, next to the existing stream-truncationrules:
PR (on our fork): apple-ouyang#2
The PR also folds in two wordings we currently carry as a local patch against
0.1.7:unexpected EOF(Node's HTTP client for a truncated response body) andremote error: tls: bad record MAC(Go for a corrupted TLS record).Note on the retry budget (separate from this fix)
Classification only decides whether a failure is retryable. Normal mode still
defaults to
maxRetries: 5with 500 ms → 10 s backoff (≈15 s total). On2026-10-03 a local gateway stayed unavailable for ~77 s, so those five retries
were exhausted and the turn still failed. Deployments behind a local gateway can
raise
retryPolicy.maxRetries/backoff.maxDelayMsin their provider profile;we set 8 retries and 1 s → 30 s locally. That is a configuration choice rather
than an upstream default, but it may be worth documenting alongside the retry
executor.
Related
PI_AI_ERROR.stream_read_error) and [Bug] Node >=24 default HTTP/2 fetch corrupts large request bodies against api.deepseek.com: TRANSPORT retry storms (TLS bad record mac + ERR_HTTP2_INVALID_SESSION) #4447 (DeepSeek H2 upload pathbad record mac).All reactions