Skip to content

fix(openai): tolerate object-form tool-call arguments in streaming#1822

Merged
gold-silver-copper merged 2 commits into
0xPlaygrounds:mainfrom
xavierforge:fix-openai-object-arguments
Jun 2, 2026
Merged

fix(openai): tolerate object-form tool-call arguments in streaming#1822
gold-silver-copper merged 2 commits into
0xPlaygrounds:mainfrom
xavierforge:fix-openai-object-arguments

Conversation

@xavierforge
Copy link
Copy Markdown
Contributor

@xavierforge xavierforge commented May 27, 2026

Fixes #1829

Problem

Some OpenAI-compatible gateways stream tool_calls[].function.arguments as a JSON object (e.g. "arguments": {}) instead of the spec-mandated JSON-encoded string ("arguments": "{}").

StreamingFunction.arguments is typed Option<String>, so serde fails with invalid type: map, expected a string.
In normalize_chunk the parse error is only logged, and the whole SSE chunk is dropped (Ok(None)), so the tool call never surfaces, and the turn ends with empty assistant text, and then the caller sees a blank response, with no error unless rig debug logs are enabled.

Minimal reproducing chunk:

{"choices":[{"index":0,"delta":{"role":"assistant","content":"","tool_calls":[{"index":0,"id":"call_x","type":"function","function":{"name":"list_dir","arguments":{}}}]},"finish_reason":null}],"usage":null}

Fix

Deserialize arguments with a tolerant helper: a string is taken verbatim; any other JSON value is re-serialized to its compact JSON-string form. null/missing stays None.
Fully backward compatible with the spec-compliant string form.

Moves the tolerant deserializer into a shared json_utils::deserialize_json_string_or_value (built on the existing value_to_json_string) and applies it to StreamingFunction.arguments via #[serde(deserialize_with = …)], so other providers can reuse it.
Adds unit tests in json_utils covering string / object / nested / null / missing.

Tests

Adds test_streaming_function_object_arguments (empty + nested object) and test_streaming_function_null_arguments (null / missing).
All existing streaming tests still pass.

Copy link
Copy Markdown
Contributor

@Shaurya-Sethi Shaurya-Sethi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the clear write-up and focused fix.

I've verified the deserialization failure on main using the minimal json from your pr (object-form "arguments": {} fails with invalid type: map, expected a string), but haven't reproduced it against a live openai-compatible gateway.

Before merge

  1. Issue + real repro : Please open an issue for this bug on the actual OpenAI-compatible gateway that emits object-form arguments in streaming (provider name, endpoint if relevant, and steps to reproduce). Link it from this PR (Fixes #<n>).

The new unit tests are a good regression guard, but we will still need a reproducible report on the actual gateway.

  1. Deduplication : As noted inline on de_tool_call_arguments, please consider reusing the existing json_utils::value_to_json_string helper.

Once you document and link the issue, a maintainer will likely follow-up with additional feedback before approving :)

///
/// We now accept either: a string is taken verbatim; any other JSON value is
/// re-serialized to its compact JSON-string form.
fn de_tool_call_arguments<'de, D>(deserializer: D) -> Result<Option<String>, D::Error>
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This logic duplicates json_utils::value_to_json_string in crates/rig-core/src/json_utils.rs, which already does string vs non-string normalization:

pub fn value_to_json_string(value: &serde_json::Value) -> String {
    match value {
        serde_json::Value::String(s) => s.clone(),
        other => other.to_string(),
    }
}

We could reuse that (or add a small shared deserializer in json_utils) instead of maintaining a second copy here

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for catching this.
Moved it into json_utils::deserialize_json_string_or_value (on top of the existing value_to_json_string) and applied via #[serde(deserialize_with = …)]. Added unit tests in json_utils; the streaming tests stay as integration coverage.

@xavierforge
Copy link
Copy Markdown
Contributor Author

Thanks for taking the time to verify the repro.
Opened #1829, it's an internal company gateway fronting a Mistral model (backing software unknown to me, endpoint private), but the raw SSE chunk in the issue reproduces the deserialization failure standalone.
Linked via Fixes #1829.

@gold-silver-copper gold-silver-copper force-pushed the fix-openai-object-arguments branch from eb45cc1 to ac502ea Compare June 2, 2026 01:35
@gold-silver-copper
Copy link
Copy Markdown
Contributor

Thanks for the PR! And if you want, you can mention your company in the "who's using rig" section of our readme!

@gold-silver-copper gold-silver-copper added this pull request to the merge queue Jun 2, 2026
Merged via the queue into 0xPlaygrounds:main with commit d1053e7 Jun 2, 2026
5 checks passed
This was referenced Jun 2, 2026
@xavierforge
Copy link
Copy Markdown
Contributor Author

Thanks! Happy to contribute, and thanks for the offer about the README!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: OpenAI-compatible gateway streams tool-call arguments as object, breaking streaming tool calls

3 participants