fix(lite_llm): tolerate malformed JSON in tool call arguments#5897
fix(lite_llm): tolerate malformed JSON in tool call arguments#5897Ja-Crispy wants to merge 3 commits into
Conversation
When the upstream model returns malformed JSON in tool_call.function.arguments (truncated string, unclosed object, ...), _message_to_generate_content_response raised json.JSONDecodeError and aborted the invocation before tool callbacks could recover. The crash surfaced as an unrelated 'Something went wrong' message to the user. Treat malformed JSON as a recoverable model error: log a warning with the raw arguments and pass an empty dict so the downstream tool dispatch can surface a structured error the model can retry from.
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
|
Response from ADK Triaging Agent Hello @Ja-Crispy, thank you for your contribution and for providing such a detailed PR description and comprehensive testing plan! It looks like the Contributor License Agreement (CLA) check is currently failing. Per our Contribution Guidelines, all contributions must be accompanied by a signed CLA before they can be reviewed and accepted. Could you please visit https://cla.developers.google.com/ to sign the CLA? This will help the maintainers to review and merge your changes. Thank you! |
|
@adk-bot I signed it! |
|
Hi @Ja-Crispy , Thank you for your contribution! We appreciate you taking the time to submit this pull request. Your PR has been received by the team and is currently under review. We will provide feedback as soon as we have an update to share. |
|
Hi @wyf7107 , can you please review this. |
Link to Issue or Description of Change
Problem:
When the upstream model returns malformed JSON in
tool_call.function.arguments(truncated strings, unclosed objects, partial streams committed to the message),_message_to_generate_content_responseingoogle/adk/models/lite_llm.pycallsjson.loads(tool_call.function.arguments or "{}")without a guard. The resultingjson.JSONDecodeErrorpropagates out of LiteLLM response parsing — upstream of any tool or callback boundary — so the entire invocation aborts and the user sees a generic streaming-error message rather than a recoverable tool-call retry.Solution:
Wrap the
json.loadscall in atry/except json.JSONDecodeError. On failure, log a warning that includes the tool name, the tool-call id, and the raw arguments (so operators can debug what the model emitted), and pass an empty dict asargs. The function-callPartis still constructed with the original tool name and id, so the downstream tool dispatch fires with no arguments — the resultingbad_arguments-style error from the tool layer is a recoverable signal the model can read and retry from.This matches the expected behaviour described in #5008 ("recoverable model error path and retry/fallback") and aligns with how ADK already tolerates other model-output edge cases.
Testing Plan
Unit Tests:
Added
tests/unittests/models/test_lite_llm_malformed_tool_args.pywith four test functions covering both the new tolerant path and the existing happy path:test_unterminated_string_does_not_raise— verifies the exact stack trace from [LiteLlm] JSONDecodeError on malformed tool_call.function.arguments crashes run before callbacks can recover #5008 ({"city":"unterminated) no longer raises, the resultingPart.function_call.args == {}, and a warning is logged with the raw arguments.test_unclosed_object_does_not_raise— additional malformed-JSON shape ({"a": 1, "b":) for breadth.test_well_formed_arguments_still_parse— regression that valid JSON still produces the parsed dict ({"city": "Paris", "units": "metric"}).test_empty_arguments_become_empty_dict— regression that the existingor "{}"fallback for empty strings still works.Local run:
Full
models/suite still green to confirm no regression:Manual End-to-End (E2E) Tests:
In our deployment (which uses this fix via a local patch), the same prompt that previously crashed the invocation now produces a tool-response cycle the model recovers from — the user sees the corrected tool result instead of the generic streaming error. Happy to provide additional repro if helpful.
Checklist