Skip to content

Decide the generator's empty-response behaviour #178

Description

@NathanTarbert

packages/outpost/ai/src/generator.ts was changed on #170's branch to throw when the model returns no usable text:

const responseText = extractResponseText(message.content);
if (!responseText.trim()) {
    throw new Error('Model response contained no usable text');
}

Previously it returned an empty string. The change came in alongside a genuine fix — the generator only read message.content[0], so a multi-block response silently lost every block after the first — but the throw is a separate, unrequested behaviour change and deserves its own decision.

What it changes

Every caller of the generator is affected, not just the support-response path. An empty or tool-only model response now becomes a thrown error, which becomes a failed job, which after MAX_JOB_ATTEMPTS walks to DEAD_LETTER.

The case for it

Fail-loud is usually right. Publishing an empty response to a customer is worse than failing visibly, and an empty completion nearly always means something upstream is wrong — a bad prompt, a refusal, a truncated stream, a tool-only turn being treated as final.

The case against it

  • It converts a recoverable oddity into a terminal job failure. Retrying an empty completion is often the right move, and three retries then dead-letter is a heavy response.
  • A tool-only response is legitimately empty of text. If any pipeline path uses tool calls, this throws on a correct response.
  • The blast radius is every generator caller, decided in a PR about something else.

What to decide

  1. Is throwing correct, or should an empty response be retried, or returned and handled by the caller?
  2. If it throws, should it be a distinct error type so the queue can classify it as retryable rather than dead-lettering?
  3. Should the check be !responseText.trim(), or narrower — e.g. only when there are no content blocks at all, so a tool-only turn passes through?

Small change, real reach. Worth an explicit call rather than inheriting it from a multi-block bug fix.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: aiAI response pipeline, groundedness, confidenceroadmapTracked on the Outpost roadmaproadmap: nextRoadmap horizon: after launch path clears

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions