[llm-pi-ai] Upstream "Provider returned an empty response" (OpenRouter) falls through classifyPiAiError to non-retryable PI_AI_ERROR

[PixelAlchemist4]

Environment

• dsh desktop 0.2.0-rc.2 (Windows), dsh-llm-pi-ai serving an OpenRouter route (api: openai-completions)
• dsh-llm-retry is active through the stock dsh-base bundle registration — confirmed at runtime by the presence of llm/retry-started session events. So this is not a missing-plugin issue: the retry executor runs, the failure is in classification.

Symptom

When OpenRouter relays an upstream empty response, the turn ends with an agent error and no retry attempt:

attempt t3s2 → error: PI_AI_ERROR Provider returned an empty response
AGENT ERROR t3: Provider returned an empty response

Seven attempt-level failures logged across Sep 29–30, all on the same route. dsh-llm-retry never fires for them.

Root cause

"Provider returned an empty response" is the upstream provider's error text, relayed verbatim by pi-ai's OpenAI-completions adapter into the terminal error event's errorMessage. classifyPiAiError() (packages/llm/llm-pi-ai/src/stream.ts) has no rule that matches the wording, so it falls to the catch-all:

  ...
  return 'PI_AI_ERROR'

PI_AI_ERROR is not in DEFAULT_RETRYABLE_CODES (EMPTY_RESPONSE, RATE_LIMIT, SERVER, TIMEOUT, TRANSPORT), so recover() short-circuits at !policy.retryableCodes.includes(failure.code) and the request is never retried.

Note the inconsistency: mapStopReason() classifies a locally detected zero-content stop as EMPTY_RESPONSE — retryable by default. Two paths that both mean "the model produced nothing" disagree; one retries, the other silently ends the turn.

Why this should be retryable

EMPTY_RESPONSE_CODE's own docstring (packages/llm/llm/src/types/error.ts) states the principle:

The attempt produced nothing durable, so retry policy treats it as safe to repeat.

An upstream-relayed empty response is exactly "nothing durable" — the same transient failure as a mid-stream truncation, just worded by a different layer.

Proposed fix

Narrow: one more rule in classifyPiAiError:

if (/provider returned an empty response/i.test(message)) return 'EMPTY_RESPONSE'

Structural (preferred): this is the fourth report on the same regex chain — #4361 (finish_reason: network_error), #6595 (localized timeout text), #7691 (textual HTTP 5xx). Each report adds new wording, which suggests text enumeration alone can't keep up. A wording-independent rule would end the whack-a-mole: when a terminal error arrives and the attempt emitted zero content blocks, classify it as EMPTY_RESPONSE regardless of message text. That implements the "nothing durable → safe to repeat" principle at the layer that actually knows how much was produced.

Workaround until then

Provider-level config, no code change required:

retryPolicy:
  mode: normal
  maxRetries: 5
  retryableCodes: [EMPTY_RESPONSE, RATE_LIMIT, SERVER, TIMEOUT, TRANSPORT, PI_AI_ERROR]

Two caveats worth knowing before copying this: retryableCodes replaces the default list rather than merging with it, and the PI_AI_ERROR catch-all also carries pending/deferred endings that mapStopReason()'s docstring marks as intentionally non-retryable — so it is best scoped to the route that actually needs it.

[argszero]

Thanks for the attempt log — the seven PI_AI_ERROR attempts across two days with no llm/retry event between them is the whole story, because it rules out the executor being absent and points straight at the decision. I read the path end to end; the mechanism is exactly as you suspected, with one detail worth adding.

The chain, link by link (master as of today)

1. The provider's wording arrives verbatim. The pi-ai OpenAI-completions adapter catches the empty response and stores the provider's own text on the terminal error event.

2. classifyPiAiError cannot place it. packages/llm/llm-pi-ai/src/stream.ts:42 is a chain of wordings — auth, quota, 429, 413, 400, 5xx, timeout, stream truncation, transport — and none of them matches Provider returned an empty response, so the text falls to the catch-all at :67:

   return 'PI_AI_ERROR'

3. Nothing downstream re-classifies. mapStopReason (stream.ts:88) sends every stopReason === 'error' through classifyPiAiError at :125, and the finish chunk is { type: 'finish', reason: { kind: 'error', failure } }. The agent loop forwards finish.failure verbatim into agent/request-error.

4. The executor decides on the code alone. packages/llm/llm-retry/src/index.ts:

   } else if (!policy.retryableCodes.includes(failure.code)) {
     return next()
   }

5. PI_AI_ERROR is not in the default table. packages/llm/llm/src/retry-policy.ts:16-22:

   const DEFAULT_RETRYABLE_CODES = Object.freeze([
     EMPTY_RESPONSE_CODE, 'RATE_LIMIT', 'SERVER', 'TIMEOUT', 'TRANSPORT',
   ])

So the attempt is declined before any llm/retry event could be appended — which is precisely the signature your log shows.

The asymmetry that makes this a gap rather than a policy

mapStopReason already has a branch for this semantic situation: a terminal stop with zero content blocks is mapped to EMPTY_RESPONSE_CODE (stream.ts:104-112), and EMPTY_RESPONSE_CODE is in the default table. Its comment says the reason outright — "The attempt produced nothing durable, so retry policy treats it as safe to repeat."

The same thing happened to you, and the harness declined to retry it — the only difference is who noticed. When the adapter notices locally it is retryable; when the provider reports it as an error and the classifier cannot place the wording, it is not. Identity of the code decides, not the property of the attempt, and the code is chosen by a wording table with a catch-all that means both "transient, unnamed" and "terminal by design".

Why the fix cannot be one line in the table

There are exactly three producers of PI_AI_ERROR, all in stream.ts: the catch-all at :67, and two deliberate terminal states — case 'pending' at :113 and case 'deferred' at :117, documented as "non-retryable PI_AI_ERROR failures" in the mapStopReason doc comment. Adding PI_AI_ERROR to DEFAULT_RETRYABLE_CODES would silently make a pending/deferred stream retryable too, turning two designed end-states into retry loops. Any correct fix has to discriminate per attempt, not per code.

What is available to discriminate on at the seam is the one thing the provider gave you and the adapter preserved: the attempt produced zero reader-visible chunks. A failed attempt that streamed text before failing is a different animal from one that never emitted anything, and only the latter is safe to repeat unconditionally.

A plugin that closes it

I have published the misfire as a plugin rather than leaving it as an analysis, since the seam (llm/stream, a public waterfall) is the same one dsh-llm-retry sits behind:

@argszero/cordis-plugin-empty-response-retry@0.1.0 — https://www.npmjs.com/package/@argszero/cordis-plugin-empty-response-retry
source: https://github.com/argszero/cordis-plugin-empty-response-retry

npm install @argszero/cordis-plugin-empty-response-retry

It observes the llm/stream waterfall and, on an error finish that satisfies all three of —

• failure.code is in codes (default ['PI_AI_ERROR']),
• the attempt produced no reader-visible chunk at all,
• the message is the forwarded provider original (not anything the harness authored),

— rewrites the terminal chunk's failure.code to the by-value EMPTY_RESPONSE_CODE, the exact code the local zero-output branch already writes. Nothing else about the finish is touched, so the turn renders, logs and replays as before; the only difference is that dsh-llm-retry's existing predicate now accepts the step.

pending and deferred are untouched by construction, not by a message heuristic: the plugin only widens the catch-all path, and the two designed terminal states keep the code they were given.

Configuration (all validated at mount time):

key	default	meaning
mode	'error'	'warn' logs the verdict without changing behaviour; 'off' forwards everything unchanged.
codes	['PI_AI_ERROR']	Codes whose zero-output attempts are widened.
exceptMessages	['^pi-ai ']	Regex sources; a matching message is left exactly as the adapter wrote it. Tighten it to your own wording (Provider returned an empty response) if you would rather the rule be narrower than the catch-all.
targetCode	'EMPTY_RESPONSE'	Code written on a widened finish.

Since you are the one with the data, start with mode: warn: the log line names the route, the adapter's code and the provider's message, so you can measure how often your traffic actually hits the rule before it changes behaviour. It does not read or write retryableCodes / maxRetries / backoff — those stay the provider's policy.

Verified against 0.1.6-alpha.2, 0.1.7-rc.2 and 0.2.0-rc.2 (the three published lines), with the real dsh-llm-pi-ai + a stubbed provider, and the real dsh-llm-retry reaching a {kind:'retry'} decision. An unmounted control run still ends in the terminal PI_AI_ERROR your log shows.

If you would rather this landed in core as a widened classifier rule for the empty-response wording, that is a reasonable alternative — the discriminator above (zero output produced) is the part I would insist on either way, because the code alone does not carry it.