Skip to content

fix: #3308 reject chat custom tool calls explicitly#3309

Merged
seratch merged 3 commits intoopenai:mainfrom
Aphroq:fix/chatcmpl-custom-tool-call-error
May 10, 2026
Merged

fix: #3308 reject chat custom tool calls explicitly#3309
seratch merged 3 commits intoopenai:mainfrom
Aphroq:fix/chatcmpl-custom-tool-call-error

Conversation

@Aphroq
Copy link
Copy Markdown
Contributor

@Aphroq Aphroq commented May 9, 2026

Summary

  • Gate Chat Completions custom tool-call rejection behind strict_feature_validation so the default behavior stays compatible.
  • Raise UserError for unsupported custom tool calls in both non-streaming and streaming Chat Completions strict mode.
  • Track ignored streamed custom tool-call indexes so later chunks without type do not become malformed function-call items.
  • Add regression coverage for default compatibility, strict-mode rejection, mixed function/custom calls, and streamed custom chunks.

Test plan

  • uv run pytest tests/models/test_openai_chatcompletions_converter.py -k custom_tool_call
  • uv run pytest tests/models/test_openai_chatcompletions_converter.py -k "custom_tool_call or mixed_custom"
  • uv run pytest tests/models/test_openai_chatcompletions_stream.py -k custom_tool_call
  • uv run pytest tests/models/test_openai_chatcompletions.py -k custom_tool_call
  • uv run pytest tests/models/test_openai_chatcompletions_converter.py tests/models/test_openai_chatcompletions_stream.py tests/models/test_openai_chatcompletions.py
  • bash .agents/skills/code-change-verification/scripts/run.sh

Issue number

Closes #3308

Checks

  • I've added new tests (if relevant)
  • I've added/updated the relevant documentation
  • I've run make lint and make format
  • I've made sure tests pass

@github-actions github-actions Bot added bug Something isn't working feature:chat-completions labels May 9, 2026
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1b2c3cbed8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/agents/models/chatcmpl_converter.py Outdated
@Aphroq Aphroq changed the title fix(models): reject chat custom tool calls explicitly fix: #3308 reject chat custom tool calls explicitly May 9, 2026
@Aphroq
Copy link
Copy Markdown
Contributor Author

Aphroq commented May 9, 2026

@codex review

@chatgpt-codex-connector
Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🚀

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Copy Markdown
Member

@seratch seratch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix. One compatibility concern: this changes the released Chat Completions behavior from silently ignoring custom tool calls to always raising. That is safer, but it is still a behavior change for users/providers that currently rely on the old behavior.

Can we gate this behind strict_feature_validation instead?

Suggested shape:

  • Pass OpenAIChatCompletionsModel._strict_feature_validation into both Converter.message_to_output_items() and ChatCmplStreamHandler.handle_stream().
  • Raise only when strict_feature_validation=True.
  • Keep the default False behavior compatible with today’s behavior.
  • For streaming, please make sure ignored custom tool call indexes stay ignored across later chunks where type may be omitted, so we do not accidentally build a malformed ResponseFunctionToolCall.

Since existing strict feature validation errors in this model use UserError, I think this should probably use UserError too rather than AgentsException.

@seratch seratch marked this pull request as draft May 10, 2026 01:02
@Aphroq
Copy link
Copy Markdown
Contributor Author

Aphroq commented May 10, 2026

Handled. PR updates:

  • Converter.message_to_output_items() now accepts strict_feature_validation, defaulting to False, so default Chat Completions behavior remains compatible and continues ignoring custom tool calls.
  • OpenAIChatCompletionsModel passes _strict_feature_validation into both the non-streaming converter and streaming ChatCmplStreamHandler.handle_stream().
  • With strict_feature_validation=True, both non-streaming and streaming paths now reject custom tool calls with UserError.
  • The default streaming compatibility path tracks ignored custom tool-call indexes. If later chunks omit type, the same index stays ignored and is not accidentally assembled into a ResponseFunctionToolCall.

Extra edge coverage:

  • Converter default compatibility: custom tool call returns no output, matching the old behavior.
  • Converter strict mode: custom tool call raises UserError.
  • Converter strict mixed case: function + custom does not partially convert the function while hiding the custom call.
  • Model get_response strict mode: confirms OpenAIChatCompletionsModel._strict_feature_validation reaches the converter.
  • Streaming strict mode: custom delta raises UserError.
  • Streaming default mode: custom index followed by no-type chunks remains ignored and does not emit function-call events or final output.

@Aphroq Aphroq force-pushed the fix/chatcmpl-custom-tool-call-error branch from f854554 to 962a2cb Compare May 10, 2026 06:18
@Aphroq Aphroq marked this pull request as ready for review May 10, 2026 06:20
@Aphroq
Copy link
Copy Markdown
Contributor Author

Aphroq commented May 10, 2026

@codex review

@seratch seratch added this to the 0.17.x milestone May 10, 2026
@chatgpt-codex-connector
Copy link
Copy Markdown

Codex Review: Didn't find any major issues. You're on a roll.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@seratch seratch merged commit 479640e into openai:main May 10, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working feature:chat-completions

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Chat Completions converter silently drops custom tool calls

2 participants