Skip to content

feat: Adding support for streaming "create responses" response - #598

Merged
flsh86 merged 1 commit into
softwaremill:masterfrom
wookievx:openai-responses-streaming-support
Aug 25, 2026
Merged

feat: Adding support for streaming "create responses" response#598
flsh86 merged 1 commit into
softwaremill:masterfrom
wookievx:openai-responses-streaming-support

Conversation

@wookievx

@wookievx wookievx commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

Brief summary of changes:

  1. Introducing explicit modeling of streaming /response model, as defined by: https://developers.openai.com/api/docs/guides/streaming-responses.
  2. Supporting fs2/zio/akka/ox.
  3. Handling observed discrepancy from the official documentation: https://developers.openai.com/api/reference/resources/beta/subresources/responses/streaming-events#response.function_call_arguments.done (no name in actual responses, removed from the Scala model)
  4. Added newest generation of models (gpt-5.6-x, to documented model list)

@flsh86 flsh86 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wookievx
wookievx force-pushed the openai-responses-streaming-support branch from 657ba9c to 6832988 Compare August 24, 2026 09:58
@adamw

adamw commented Aug 24, 2026

Copy link
Copy Markdown
Member

Review findings:

Bugs

  1. ChatRequestBody.scala:360ChatCompletionModel.GPT56Terra has wire value "gpt-5.4-terra" instead of "gpt-5.6-terra" (the ResponsesModel counterpart is correct).
  2. ChatRequestBody.scala:361ChatCompletionModel.GPT56Sol has wire value "gpt-5-6-sol" instead of "gpt-5.6-sol".
  3. ResponsesResponseBody.scala:520 — unknown content-part types are not covered by the forward-compat fallback: response.content_part.added/.done are known event types, so an unmodelled part type (e.g. a future output_audio) fails decoding and errors the whole stream — the case the Unknown event/item fallbacks were added to prevent.
  4. ChatRequestBody.scala:354 — none of the 8 new ChatCompletionModel case objects are in ChatCompletionModel.values, so e.g. "gpt-5.2" in a response decodes to CustomChatCompletionModel("gpt-5.2") and never matches the constant. (These chat-model additions also bypass the model_update_scripts workflow — likely how the two typos above got in.)
  5. ResponsesModel.scala:102GPT56Terra and GPT56Sol are missing from ResponsesModel.values (only GPT56Luna was added), so they never decode from responses.

Cleanups

  1. ResponsesStreamEvent.scala:450 — nothing ties the sealed-trait cases to the hand-maintained KnownTypes sets. Adding a new event but forgetting the KnownTypes entry compiles and passes tests, yet the event always decodes as Unknown in production. A wire-type → leaf-decoder Map (with KnownTypes := map.keySet), or a test enumerating the ADT cases against the set, would close this.
  2. ResponsesStreamEventFixture.scala:64jsonCompletedWithFullUsage splices usage into jsonResponseBasic by substring surgery, producing two top-level "usage" keys; the test passes only because circe keeps the last one. deepMerge on parsed JSON would be robust.
  3. ResponsesStreamEvent.scala:649ResponsesStreamEvent.LogProb/TopLogProb duplicate OutputContent.LogProb/TopLogProb; the same wire object surfaces as two Scala types depending on which event delivered it. OutputTextDelta/Done could reuse OutputContent.LogProb.
  4. OpenAIManualCodecs.scala:301responsesStreamEventDispatch rewrites the JSON object and re-dispatches through the derived sum decoder on every event, including every output_text.delta. A precomputed wire-type → decoder Map would avoid the rewrite and the double dispatch (and subsume KnownTypes, see 6).
  5. OpenAIResponsesOxStreamingIntegrationSpec.scala:87 — the function-tool-call integration test has no maxOutputTokens; the sibling test caps at 16. A small cap would keep the paid call cheap.

@wookievx

Copy link
Copy Markdown
Contributor Author

Resolved the issues

@wookievx
wookievx force-pushed the openai-responses-streaming-support branch from 6832988 to 6074c81 Compare August 25, 2026 10:06
@flsh86
flsh86 merged commit 6a489fe into softwaremill:master Aug 25, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants