Skip to content

fix(models): surface content-filter refusals in non-streaming Chat Completions - #4159

Closed
LeSingh1 wants to merge 1 commit into
openai:mainfrom
LeSingh1:fix/chatcmpl-content-filter-refusal
Closed

fix(models): surface content-filter refusals in non-streaming Chat Completions#4159
LeSingh1 wants to merge 1 commit into
openai:mainfrom
LeSingh1:fix/chatcmpl-content-filter-refusal

Conversation

@LeSingh1

@LeSingh1 LeSingh1 commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Summary

OpenAIChatCompletionsModel.get_response never inspected finish_reason. Some providers — Azure OpenAI content filtering, Bedrock-backed Chat Completions proxies — signal a safety block only via finish_reason == "content_filter" with an empty message and no refusal field. Converter.message_to_output_items then produces zero output items, so the caller sees an indistinguishable "empty turn" and agent loops retry fruitlessly against a block that will never clear.

Both sibling paths already handle this. The streamed handler synthesizes a ResponseOutputRefusal (chatcmpl_stream_handler.py), and so does LitellmModel.get_response (#3769). This brings the non-streaming OpenAI Chat Completions path into agreement with them by setting message.refusal before conversion, and only when the turn is genuinely empty — no content, no provider-supplied refusal, and no tool calls — so no existing successful shape changes.

Test plan

New tests/models/test_openai_chatcompletions_content_filter.py, mirroring the existing test_litellm_content_filter.py:

  • a refusal is synthesized for an empty content-filtered turn
  • real content, a provider-supplied refusal, and tool calls are each left untouched
  • finish_reason="stop" is unaffected in both the text and empty cases

Fails on main (AssertionError: expected a refusal item, got: []), passes with the fix. Full tests/models/ suite: 646 passed. .agents/skills/code-change-verification/scripts/run.sh passed.

Issue number

N/A — found while auditing provider parity between the streaming and non-streaming Chat Completions paths.

Checks

  • I've added new tests, if relevant
  • I've run .agents/skills/code-change-verification/scripts/run.sh
  • I've confirmed all verification steps pass
  • If using Codex, I've run /review before submitting this PR

@seratch seratch left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the contribution. The behavior and fix are valid, but the impact description is slightly too broad: an empty content-filtered response does not always drive the agent loop into retries. For plain-text agents, the current runner completes successfully with an empty string; structured-output agents retry until the turn limit.

Please update the source comment, test docstring, and PR description to state that the lost refusal signal produces an empty final output for plain-text runs and repeated retries for structured-output runs. No runtime implementation change is needed.

@seratch

seratch commented Aug 5, 2026

Copy link
Copy Markdown
Member

#4188 includes your contribution; thanks again for your efforts here.

@seratch seratch closed this Aug 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants