Skip to content

fix(provider): inject placeholder text for empty assistant wire messages on openai-compatible#38270

Open
codeg-dev wants to merge 1 commit into
anomalyco:devfrom
codeg-dev:codeg/fix-empty-assistant-wire
Open

fix(provider): inject placeholder text for empty assistant wire messages on openai-compatible#38270
codeg-dev wants to merge 1 commit into
anomalyco:devfrom
codeg-dev:codeg/fix-empty-assistant-wire

Conversation

@codeg-dev

Copy link
Copy Markdown

Issue for this PR

Fixes #37946

Related: #37651 / #37624 (empty reasoning parts — complementary; that fix skips empty reasoning parts, while this PR covers assistant wire messages that still end up with no non-empty text at all, e.g. aborted reasoning-only turns and step-start split pieces). Also related: #31046 (empty text parts), #28507, #37852, #37372.

Type of change

  • Bug fix

What does this PR do?

@ai-sdk/openai-compatible serializes assistant messages without any non-empty text as content: "" or null. Strict providers (Kimi/Moonshot via chat/completions) reject the whole request with 400 "the message at position N with role 'assistant' must not be empty". Once a session contains such a turn it is permanently bricked: every retry fails the same way and appends another empty artifact message.

Shapes that produce an empty assistant wire message:

This PR runs a post-pass over the final ModelMessage list in toModelMessagesEffect: when the target model uses @ai-sdk/openai-compatible, any assistant message lacking non-empty text gets a single-space text part unshifted, so every split piece is covered individually.

Why inject instead of dropping the empty message: dropping an assistant turn that carries tool calls would break tool-call/tool-result pairing, and dropping a reasoning-only turn would discard interleaved reasoning_content that providers like Kimi use as history context. A single-space placeholder keeps history semantics intact while satisfying the non-empty-content requirement.

Scoped to openai-compatible on purpose: OpenAI and Anthropic accept textless tool-call turns, and Anthropic signed thinking is handled before conversion.

How did you verify your code works?

  • 7 new regression tests in test/session/message-v2.test.ts (tool-call-only, step-start split pieces, empty-text-only, reasoning-only passthrough, anthropic unaffected, step-start-only still dropped, existing text untouched): 43/43 pass.
  • tsgo --noEmit clean.
  • Replayed a bricked Kimi K3 session containing an aborted reasoning-only turn through a patched build: the follow-up request succeeded (previously 400 at the artifact position, and the position no longer advances on retry).

Screenshots / recordings

Not applicable.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

Prepared with AI assistance (OpenCode agent); reviewed and verified by the author before submission.

…ges on openai-compatible

@ai-sdk/openai-compatible serializes assistant messages without any
non-empty text as content: "" or null — tool-call-only turns, the
per-step pieces produced by step-start splitting, and aborted
reasoning-only turns all hit this. Strict providers (Kimi/Moonshot via
chat/completions) reject the whole request with 400 "the message at
position N with role 'assistant' must not be empty", permanently
bricking the session as each failed retry appends another empty
artifact message.

Run a post-pass over the final ModelMessage list in
toModelMessagesEffect when the target model uses
@ai-sdk/openai-compatible and unshift a single-space text part into
any assistant message lacking non-empty text, so every split piece is
covered individually. Scoped to openai-compatible: OpenAI and
Anthropic accept textless tool-call turns, and Anthropic signed
thinking is handled before conversion.

Fixes anomalyco#37946

Tests: 43/43 in test/session/message-v2.test.ts (7 new: tool-call-only,
step-start split pieces, empty-text-only, reasoning-only passthrough,
anthropic unaffected, step-start-only still dropped, existing text
untouched)
@github-actions

Copy link
Copy Markdown
Contributor

The following comment was made by an LLM, it may be inaccurate:

I found a related PR that's worth reviewing:

PR #33899 - fix(provider): ensure non-empty assistant content for openai-compatible (#33280)
#33899

This PR appears to address a similar issue of ensuring non-empty assistant content for the openai-compatible provider. It may have already tackled part of this problem or provide relevant context for the current PR.

The other results are either the current PR itself (#38270) or tangentially related PRs about empty content, reasoning parts, and provider fixes.

You should verify whether PR #33899 already solves the issue or if this PR (#38270) is extending/improving upon that previous fix.

@codeg-dev

Copy link
Copy Markdown
Author

Verified the overlap with #33899 — thanks for surfacing it. Short version: #33899 does not fix #37946; this PR covers a superset of #33899's cases.

#33899 (still open, unmerged) patches ProviderTransform.message in src/provider/transform.ts and injects the single-space text only when the assistant message has tool calls but no text — its guard is if (hasText || !hasToolCall) return msg. Assistant messages with no tool calls and no text pass through untouched.

#37946's failing shape is exactly that untouched case: an aborted turn persisted with only step-start / reasoning / patch parts — no text, no tool calls — replays as content: [] (or ""), and strict providers reject the whole request with 400. #33899's !hasToolCall early-return leaves it empty. (The two issues are also different failure modes: #33280 is a lenient proxy injecting a visible placeholder into the transcript; #37946 is a strict provider rejecting the request and bricking the session.)

Coverage comparison:

wire message shape #33899 this PR
tool-call-only, no text (#33280)
step-start split pieces (tool-call-only after split)
aborted reasoning-only turn, no text + no tool calls (#37946) ❌ (!hasToolCall skip)
assistant message whose only text part is ""
string content: "" ❌ (skipped) ✅ (→ " ")

Layer note: this PR runs the guard in toModelMessagesEffect, which produces the list later passed to ProviderTransform.message in the request path; #33899 runs inside ProviderTransform.message. If both were merged they would not double-inject (both check for existing non-empty text first), but one guard at one layer would be cleaner. Happy to move this guard into ProviderTransform.message with the hasToolCall requirement dropped — i.e. #33899's location with this PR's wider condition — if maintainers prefer that layer. Either way, fixing #37946 requires covering the no-tool-call empty case that #33899 currently skips.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Aborted assistant turn persisted with zero content bricks the session: empty assistant message replayed to provider (400 "must not be empty")

1 participant