Skip to content

openai-responses provider drops entire SSE stream when gateway omits event: lines (empty content, zero usage) #434

Description

@initializ-mk

Problem

The openai-responses provider's SSE parser (readStream, forge-core/llm/providers/responses.go) routes events solely on the SSE event: line:

if after, ok := strings.CutPrefix(line, "event: "); ok {
    currentEvent = after      // ← only ever set here
    continue
}
data, ok := strings.CutPrefix(line, "data: ")
...
switch currentEvent { case "response.output_text.delta": ... case "response.completed": ... }

The OpenAI Responses API sends both an event: line and a type field inside the data: payload. But event: is optional per the SSE spec, and some gateways omit it — notably the Bedrock openai-sigv4 shim (Kong route /bedrock/openai-sigv4/v1/responses), which streams spec-legal SSE carrying the type only as a JSON field:

data: {"content_index":0,"delta":"Loading","type":"response.output_text.delta"}
...
data: {"response":{...,"usage":{"input_tokens":76,"output_tokens":73,"total_tokens":149}},"type":"response.completed"}

With no event: lines, currentEvent stays "", the switch matches no case, and every frame is dropped — text deltas and the terminal response.completed (which carries usage). Result: empty assistant message, zero tokens, empty finish reason.

Field evidence

Observed on a staging agent (openai-responses / openai.gpt-5.4 via the Bedrock gateway):

  • llm_call audit: input_tokens:0, output_tokens:0, tokens_unavailable:true, total_input_tokens:0 on a 619ms call.
  • Runtime log: {"finish_reason":"","msg":"llm response"}empty finish reason is the smoking gun, since response.completed is the only handler that sets one.
  • invocation_complete: all token totals zero; the user got an empty answer.

The raw stream (captured via curl) confirms the gateway returns correct content + usage:{input_tokens:76,output_tokens:73,total_tokens:149} — forge simply never parses it.

Fix

Dispatch on the payload type field, falling back to the event: line only when the payload has none (authoritative + present in both dialects; matches how the OpenAI SDKs route):

eventType := currentEvent
var typed struct{ Type string `json:"type"` }
if json.Unmarshal([]byte(data), &typed) == nil && typed.Type != "" {
    eventType = typed.Type
}
switch eventType { ... }

Also bump bufio.Scanner's 64KB line cap — the terminal response.completed frame embeds the full output[] on one data: line and overruns it on large answers, aborting the stream mid-parse.

Acceptance

  • A stream with no event: lines (type in data.type) parses text + usage correctly.
  • The dual-dialect stream (both event: and type) still works (real OpenAI) — no regression.
  • A >64KB terminal frame parses without a scanner overflow.

Related / out of scope

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions