Skip to content

with_structured_output returns None when MALFORMED_FUNCTION_CALL happens without raising errors #1207

@pokutuna

Description

@pokutuna

When using with_structured_output(), occasionally the return value becomes None unexpectedly. This happens even with function calling that normally succeeds, where the Gemini API sometimes returns MALFORMED_FUNCTION_CALL responses. This silent failure occurs without any error indication, making it extremely difficult to diagnose the root cause during development.

Through extensive debugging of this intermittent issue, I confirmed that the Gemini API sometimes probabilistically returns MALFORMED_FUNCTION_CALL responses. The investigation revealed that these responses have the following structure:

{
  "content": "",
  "response_metadata": {
    "finish_reason": "MALFORMED_FUNCTION_CALL",
    "finish_message": "Malformed function call: print(default_api.Output(...",
    "tool_calls": [],
    "invalid_tool_calls": [],
    ...
  }
  ...
}

When with_structured_output() is used, PydanticToolsParser and JsonOutputKeyToolsParser are connected to the chain. These parsers return None or empty lists when tool_calls is empty, without detecting the underlying MALFORMED_FUNCTION_CALL error.

When using with_structured_output(..., method="json_mode"), PydanticOutputParser and JsonOutputParser are used instead. These parsers properly throw exceptions when content parsing fails.

However, json_mode doesn't support some JSON Schema features, so developers often must use the default method despite its error handling issues.

Suggestion: Throw an error when MALFORMED_FUNCTION_CALL occurs instead of returning None

The implementation could be as simple as using a custom parser wrapper, for example:

class MalformedFunctionCallError(ChatGoogleGenerativeAIError):
    pass

class StructuredOutputParserWithErrorHandling:
    def __init__(self, parser):
        self.parser = parser

    def parse(self, ai_message):
        if (hasattr(ai_message, 'response_metadata') and
            ai_message.response_metadata.get('finish_reason') == 'MALFORMED_FUNCTION_CALL'):
            finish_message = ai_message.response_metadata.get('finish_message', 'Unknown error')
            raise MalformedFunctionCallError(finish_message)

        return self.parser.parse(ai_message)

While the implementation seems not difficult, it would affect structured output behavior overall, so I haven't prepared a Pull Request and would like to wait for discussion from maintainers.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions