Provider calls have no timeout and no retry, so a 429 fails the build

[royalpinto007]

Problem

Provider calls have no timeout and no retry.

OpenAICompatibleProvider.complete in src/providers/openai-compatible.ts:

const res = await fetch(`${this.config.baseUrl}/chat/completions`, {
  method: "POST",
  headers: { ... },
  body: JSON.stringify({ ... }),
});

No signal, no AbortController, no timeout of any kind, and src/providers/anthropic.ts should be checked for the same. Node's fetch will wait a very long time on a stalled connection, so a hung upstream means a hung CI job that eventually dies on the runner's own timeout with no useful message.

There is no retry either. A 429 or a 503 throws immediately, and as described in the sibling issue about per-case error handling, that throw currently takes the whole run with it. Nothing reads the Retry-After header. mapPool in src/runner.ts dispatches up to concurrency requests with no pacing between them, so raising --concurrency makes rate limiting more likely, not less.

Why it matters

Rate limiting is the normal operating condition for a tool that fires a burst of LLM API calls, not an edge case. A gate that fails in CI because the provider was briefly busy trains everyone to rerun the job until it goes green, which is precisely the habit that lets a real regression through.

Suggested approach

1. Add a per-request timeout with AbortSignal.timeout(ms) (available on Node 18, which engines already requires). Default to something like 60s, configurable per suite and via a CLI flag.
2. Add bounded retry with exponential backoff and jitter for 429, 500, 502, 503, 504, and network-level errors. Honor Retry-After when present. Cap attempts, default 3, and make it configurable.
3. Never retry 4xx other than 429. A 400 is a bad request and retrying it just costs time.
4. Put the retry logic in one shared helper so openai-compatible.ts and anthropic.ts cannot drift, and so the mock provider can exercise it.
5. Record what happened: add an attempts field on ProviderResponse and surface a warning in the terminal reporter when a case needed retries. A run that only passed after five retries is information the user wants.
6. Consider pacing in mapPool (a minimum interval between dispatches) so higher concurrency does not immediately trigger the thing retry exists to survive.
7. Tests: a fake fetch returning 429 then 200, a fake fetch that never resolves against a short timeout, and an assertion that a 400 is not retried.

Done when

• No provider call can hang indefinitely.
• Transient failures are retried with backoff and Retry-After is honored.
• Retry behavior is configurable and reported.
• Both provider adapters share one implementation.

---

If you want to take this on, comment on the issue to claim it and it will be assigned. Please keep to a maximum of 2 open claims per person at a time so other contributors get a chance.