dsh-llm-pi-ai drops Retry-After, so 503/429 retries fall back to a 500 ms local backoff

[songlairui]

Summary

The harness already models a provider-requested retry delay (LlmFailure.providerRetryAfterMs) and @deepseek-ai/dsh-llm-retry honors it — but only @deepseek-ai/dsh-llm-deepseek ever fills it in. Every route served by @deepseek-ai/dsh-llm-pi-ai (openai-responses, openai-completions) loses the response headers before the adapter sees them, so 503/429 + Retry-After: N degrades to the default exponential backoff: 5 attempts, 500 ms → 10 s, ±10% jitter.

Environment: dsh 0.1.7-rc.1, dsh headless --json over a scratch profile (dsh-base + dsh-headless) whose default model points at a local mock.

Measured

The mock answers every request with 503 + Retry-After: 5. Time between successive POSTs:

route	attempt intervals
dsh-llm-deepseek (POST /v1/messages)	+5024 / +5012 / +5015 / +5017 / +5015 ms — header honored
dsh-llm-pi-ai (POST /v1/responses)	+1 / +540 / +1100 / +2085 / +4214 / +8615 ms — local backoff

The durable events show the same split:

// deepseek route
{"type":"llm/retry","data":{"delayMs":5000,
  "failure":{"code":"SERVER","status":503,"providerRetryAfterMs":5000,"requestId":"mock-9"}}}

// pi-ai route
{"type":"llm/retry","data":{"delayMs":517.2430188586864,
  "failure":{"message":"OpenAI API error (503): {\"message\":\"503 Service Unavailable…\"}","code":"SERVER"}}}

429 behaves the same way: pi-ai classifies it as RATE_LIMIT and retries from 500 ms while ignoring Retry-After: 4.

Repro

# mock: any path → MOCK_STATUS + Retry-After, appending arrival times to attempts.log
MOCK_STATUS=503 MOCK_RETRY_AFTER=5 node mock-server.mjs &

# patch: point the default model at the mock over the openai-responses route
cat > patch.yml <<'EOF'
- id: llm-pi-ai
  config:
    providers:
      mock:
        apiKeyEnv: MOCK_API_KEY
        api: openai-responses
        baseURL: http://127.0.0.1:8799/v1
        models: [{ id: mock-1, contextWindow: 100000 }]
- id: agent-default-model
  config: { provider: mock, model: mock-1 }
EOF
MOCK_API_KEY=test dsh headless --patch ./patch.yml --json "say hello"

# swap in the native route for the control run
cat > patch-native.yml <<'EOF'
- id: llm-deepseek
  config: { baseURL: http://127.0.0.1:8799/v1, apiKeyEnv: MOCK_API_KEY }
- id: agent-default-model
  config: { provider: deepseek-official, model: deepseek-flash }
EOF

# evidence, both layers
cat attempts.log
zstd -dc ~/.dsh/sessions/*/session.v4.jsonl.zstd | grep '"type":"llm/retry"'

Why it matters

1. Hammering. A gateway that answers 503 + Retry-After: 30 is retried 0.5 s later, five times, then the turn fails.
2. 429 is risk control on OpenAI-family accounts, not just capacity. Retrying it eagerly is exactly the shape of abusive traffic. Codex CLI defaults to retry_429: false / retry_5xx: true (HTTP 429 is surfaced as "exceeded retry limit" without retrying; expose retry_429 or clarify error openai/codex#30471 is about the misleading "exceeded retry limit" wording, not about the policy), and Anthropic documents 429s that carry no retry-after at all. The harness default is more eager than either: RATE_LIMIT is in the default retryableCodes, maxRetries: 5, backoff starting at 500 ms.

Where the header is lost

dsh-llm-pi-ai streams through pi-ai with a fixed option set (profileOptions() plus temperature / maxTokens / sessionId / signal / headers) and never passes pi-ai's onResponse callback — the one call site that receives { status, headers } right after the HTTP response (@earendil-works/pi-ai/dist/api/openai-responses.js). The failure path then keeps only formatProviderError(...), a string, so classifyPiAiError() has nothing but message text to match /\b5\d\d\b/ and /\b429\b/ against. By the time LlmFailure is constructed, the header no longer exists anywhere in the process.

The parsing convention is already available in the dependency: @earendil-works/pi-ai/dist/utils/provider-retry.js → getRetryDelayMs() reads retry-after-ms, then retry-after (delta-seconds or HTTP-date) with a cap — the same shape dsh-llm-deepseek's providerError() implements.

Suggested change

Forward the header into the field that already exists, inside dsh-llm-pi-ai:

• pass onResponse through to models.streamSimple(...) (or otherwise capture the response headers for the attempt), and
• map it onto providerRetryAfterMs of the terminal failure, matching dsh-llm-deepseek's providerError(); keep 5xx → SERVER, 429 → RATE_LIMIT.

That is enough for dsh-llm-retry to wait exactly as long as the provider asked, with no new executor, no new policy surface, and no behavioral change for routes that send no header. Happy to open a PR if that is welcome — dsh-llm-mock-server is already a devDependency of the adapter, so a mocked 503 + Retry-After test looks straightforward. A documented stance on 429 (retry vs. fail fast) would be a useful companion, since the current default retries it five times from 500 ms.

Local stopgap (context, not a proposal)

Until this lands we bridge it in an out-of-tree plugin: a process-wide fetch observer records Retry-After for retryable statuses, and a prepended agent/request-error listener writes providerRetryAfterMs/status back onto the failure — leaving the waiting, the llm/retry events, the budget and cancellation to dsh-llm-retry. The 429 timings quoted above come from that bridge, and it also offers a per-code cool-down floor for header-less risk-control 429s. It works, but it is a workaround for a header that could be read where it is produced.