Skip to content

fix: preserve chat stream moderation results - #616

Merged
jbeckwith-oai merged 1 commit into
mainfrom
codex/autoimprove-20260904-3-chat-stream-moderation
Sep 4, 2026
Merged

fix: preserve chat stream moderation results#616
jbeckwith-oai merged 1 commit into
mainfrom
codex/autoimprove-20260904-3-chat-stream-moderation

Conversation

@jbeckwith-oai

Copy link
Copy Markdown
Contributor

Problem

Moderated Chat Completions can deliver the existing moderation field on streamed raw chunks, including a final moderation-only chunk with empty choices. The chat streaming helper accumulated content and usage but dropped moderation, so current snapshots and get_final_completion returned nil even though stream_raw exposed the result.

User impact

Applications using client.chat.completions.stream for moderated completions now receive the same moderation result already available through client.chat.completions.stream_raw. Unmoderated streams remain nil.

Fix

The helper now carries moderation into the initial parsed snapshot and, for later non-nil chunks, deep-dumps and coerces the field into the existing final ChatCompletion moderation type. Omitted or explicit nil later values do not erase a previously received moderation result. Raw chunks are not mutated.

Synthetic regression proof

The focused WebMock SSE test uses only fake local data through the public stream_raw and stream APIs. Before this fix, a final empty-choices moderation chunk retained nested moderation data in stream_raw while helper snapshots and final completion returned nil. After this fix, snapshots and final completion retain the same nested input/output success and error variants, boolean flags, and numeric scores. The test also covers initial moderation surviving later omitted and nil fields, and the unmoderated nil control.

Compatibility and scope

No public signatures, generated models, dependencies, transport behavior, parsing timing, event types, safety policy, tool-call accumulation, logprob accumulation, or payload limits change. Content, choices, usage, event ordering, and raw chunk values remain covered. The diff is limited to the handwritten stream helper and one focused regression test.

Verification

  • Focused regression: 3 runs, 21 assertions, 0 failures, 0 errors
  • Affected chat streaming checks: 36 runs, 234 assertions, 0 failures, 0 errors
  • Full offline suite against the current-spec mock: 1626 runs, 14450 assertions, 0 failures, 0 errors, 1 skip
  • rake lint: RuboCop inspected 2870 files with no offenses; Sorbet and 1278 RBS validations passed
  • Trusted public custom-code budget on committed candidate: 3363 of 4000 custom lines, 637 headroom
  • Dedicated bounded response deserialization and isolation security review: no new sensitive sink, logging path, policy decision, or raw-chunk mutation
  • Adversarial review: two consecutive clean rounds, each with two new independent read-only reviewers

Limitations

This is an offline synthetic regression proof. It does not claim live server observation and does not introduce moderation enforcement.

@jbeckwith-oai jbeckwith-oai added codex-maintenance Low-risk maintenance changes created by Codex autoimprove Automated repository-health improvements labels Sep 4, 2026
@jbeckwith-oai
jbeckwith-oai requested a review from a team as a code owner September 4, 2026 16:40
@jbeckwith-oai jbeckwith-oai added codex-maintenance Low-risk maintenance changes created by Codex autoimprove Automated repository-health improvements labels Sep 4, 2026
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 4, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-04T16:42:24.914861Z ba771ce PR opened
🔒 Security Review Completed 2026-09-04T16:44:10.320030Z ba771ce PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

github-actions Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Castiron custom code

✅ No new custom-code files detected.

47 mixed files remain; 0 existing customizations changed.

Compared 91d0e86ed995ba771cec1145. Generated baselines verified.

47 existing customizations unchanged
  • lib/openai.rb
  • lib/openai/client.rb
  • lib/openai/models/chat/chat_completion_message.rb
  • lib/openai/models/chat/chat_completion_message_function_tool_call.rb
  • lib/openai/models/chat/completion_create_params.rb
  • lib/openai/models/response_format_json_schema.rb
  • lib/openai/models/responses/function_tool.rb
  • lib/openai/models/responses/response.rb
  • lib/openai/models/responses/response_create_params.rb
  • lib/openai/models/responses/response_format_text_config.rb
  • lib/openai/models/responses/response_format_text_json_schema_config.rb
  • lib/openai/models/responses/response_function_tool_call.rb
  • lib/openai/models/responses/response_function_web_search.rb
  • lib/openai/models/responses/response_output_text.rb
  • lib/openai/models/responses/tool.rb
  • lib/openai/resources/beta/threads.rb
  • lib/openai/resources/chat/completions.rb
  • lib/openai/resources/files.rb
  • lib/openai/resources/responses.rb
  • lib/openai/resources/vector_stores/file_batches.rb
  • lib/openai/resources/vector_stores/files.rb
  • lib/openai/resources/webhooks.rb
  • rbi/openai/client.rbi
  • rbi/openai/models/chat/chat_completion_message.rbi
  • rbi/openai/models/chat/chat_completion_message_function_tool_call.rbi
  • rbi/openai/models/chat/completion_create_params.rbi
  • rbi/openai/models/response_format_json_schema.rbi
  • rbi/openai/models/responses/response.rbi
  • rbi/openai/models/responses/response_create_params.rbi
  • rbi/openai/models/responses/response_function_tool_call.rbi
  • rbi/openai/models/responses/response_function_web_search.rbi
  • rbi/openai/models/responses/response_output_text.rbi
  • rbi/openai/resources/chat/completions.rbi
  • rbi/openai/resources/files.rbi
  • rbi/openai/resources/responses.rbi
  • rbi/openai/resources/vector_stores/file_batches.rbi
  • rbi/openai/resources/vector_stores/files.rbi
  • scripts/castiron/README.md
  • scripts/castiron/custom_code_report.py
  • scripts/castiron/test_custom_code_report.py

7 more in the full report.

A changed generated baseline means this report cannot reliably identify which handwritten lines changed.

Inspect the custom-code diff

Download the exact patch produced by this run (requires repository access):

gh run download 33896536957 --repo openai/openai-ruby \
  --name castiron-custom-code-33896536957-1 --dir /tmp/castiron-custom-code-33896536957-1
git apply --stat /tmp/castiron-custom-code-33896536957-1/custom-code.patch
cat /tmp/castiron-custom-code-33896536957-1/custom-code.patch

Or reproduce it from an SDK checkout containing the vendored reporter:

git fetch --no-tags origin 91d0e86ed995f54361b329d7641af9f20163f27d ba771cec1145abc878a730b91f701e529e15d063
python3 scripts/castiron/custom_code_report.py report \
  --base 91d0e86ed995f54361b329d7641af9f20163f27d \
  --head ba771cec1145abc878a730b91f701e529e15d063 --fetch --require-head-hash --public \
  --out /tmp/castiron-custom-code-ba771cec1145
cat /tmp/castiron-custom-code-ba771cec1145/custom-code.patch

This is the current full custom patch for mixed files, not an attribution of only the handwritten lines changed by this PR.

Full report and patch

@openai-sdks

openai-sdks Bot commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

OkTest Summary

237/237 SDK tests passed in 8.504s for Ruby SDK PR #616.

Test results — 42 files
Test Result Time
tests/chat-completions-complex-body.test.ts ✅ Passed 152ms
tests/chat-completions-create.test.ts ✅ Passed 159ms
tests/chat-completions-stream.test.ts ✅ Passed 106ms
tests/files-content-binary.test.ts ✅ Passed 141ms
tests/files-create-multipart.test.ts ✅ Passed 204ms
tests/files-list-pagination.test.ts ✅ Passed 127ms
tests/initialize-config.test.ts ✅ Passed 160ms
tests/instance-isolation.test.ts ✅ Passed 153ms
tests/models-list.test.ts ✅ Passed 159ms
tests/responses-background-lifecycle.test.ts ✅ Passed 182ms
tests/responses-body-method-errors.test.ts ✅ Passed 347ms
tests/responses-cancel-timeout.test.ts ✅ Passed 256ms
tests/responses-cancel.test.ts ✅ Passed 185ms
tests/responses-compact-retries.test.ts ✅ Passed 228ms
tests/responses-compact.test.ts ✅ Passed 204ms
tests/responses-create-advanced-stream.test.ts ✅ Passed 153ms
tests/responses-create-advanced.test.ts ✅ Passed 164ms
tests/responses-create-disconnect.test.ts ✅ Passed 1.076s
tests/responses-create-errors.test.ts ✅ Passed 241ms
tests/responses-create-malformed-api-responses.test.ts ✅ Passed 113ms
tests/responses-create-retries.test.ts ✅ Passed 242ms
tests/responses-create-stream-failures.test.ts ✅ Passed 129ms
tests/responses-create-stream-timeout.test.ts ✅ Passed 208ms
tests/responses-create-stream-wire.test.ts ✅ Passed 1.943s
tests/responses-create-stream.test.ts ✅ Passed 83ms
tests/responses-create-terminal-states.test.ts ✅ Passed 218ms
tests/responses-create-timeout.test.ts ✅ Passed 241ms
tests/responses-create.test.ts ✅ Passed 204ms
tests/responses-delete.test.ts ✅ Passed 225ms
tests/responses-input-items-errors.test.ts ✅ Passed 150ms
tests/responses-input-items-list.test.ts ✅ Passed 147ms
tests/responses-input-items-options.test.ts ✅ Passed 137ms
tests/responses-input-tokens-count-timeout.test.ts ✅ Passed 224ms
tests/responses-input-tokens-count.test.ts ✅ Passed 182ms
tests/responses-malformed-inputs.test.ts ✅ Passed 1.604s
tests/responses-not-found-errors.test.ts ✅ Passed 245ms
tests/responses-parse.test.ts ✅ Passed 187ms
tests/responses-retrieve-retries.test.ts ✅ Passed 260ms
tests/responses-retrieve.test.ts ✅ Passed 177ms
tests/responses-stored-method-errors.test.ts ✅ Passed 548ms
tests/retry-behavior.test.ts ✅ Passed 3.19s
tests/sdk-error-shape.test.ts ✅ Passed 344ms

View OkTest run #33896496684

SDK merge (726a76666e08) · head (ba771cec1145) · base (91d0e86ed995) · OkTest (2b1bdfd25e98)

@sylvesterkaczmarek sylvesterkaczmarek left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Treating moderation as sticky stream state is the correct accumulation rule: a moderation-only terminal chunk must update the snapshot, while later omitted or nil values must not erase an earlier result. The raw-stream parity assertions cover that boundary well.

@markstuart-oai markstuart-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The stream accumulator now carries moderation into the initial snapshot and updates it only from later non-nil chunks, preserving the existing sticky stream-state semantics. The focused public-API coverage exercises final moderation-only chunks, initial values, nil/omitted updates, and raw-stream isolation. Exact-head CI is green across the Ruby test matrix, lint, types, packaging, CodeQL, and Castiron. I could not rerun the focused test locally because the required Ruby 4.0.6 toolchain is not installed on this review box.

@jbeckwith-oai
jbeckwith-oai added this pull request to the merge queue Sep 4, 2026
Merged via the queue into main with commit c2c729c Sep 4, 2026
20 checks passed
@jbeckwith-oai
jbeckwith-oai deleted the codex/autoimprove-20260904-3-chat-stream-moderation branch September 4, 2026 17:05
@openai-sdks openai-sdks Bot mentioned this pull request Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

autoimprove Automated repository-health improvements codex-maintenance Low-risk maintenance changes created by Codex

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants