Skip to content

Conversation

@paxiaatucsdedu
Copy link
Member

@paxiaatucsdedu paxiaatucsdedu commented Oct 29, 2025

Problem

Issue link: #40

1. with_structured_output() use method="function_calling" as default method. When calling Gemini 2.5 models, no function calling is returned so with_structured_output() does not work with Gemini 2.5 models.
2. If using the json_schema method in with_structured_output() , with_structured_output() was failing with a 400 error ("Please pass in correct format of request"). The root cause is that the code was passing a plain Python dict for response_format instead of proper OCI SDK objects that the GenericChatRequest API expects.

Solution

  1. Suggest in README that user should use json_schema method when the default method does not work.
  2. Updated the json_schema method to use proper OCI SDK objects:
  • ResponseJsonSchema - wraps the JSON schema with name, description, and strict mode
  • JsonSchemaResponseFormat - the proper response format type expected by the API

Testing

Now Gemini-2.5 model returns properly formatted object instead of None when using json_schema method in with_structured_output() function.
image

Fix json_schema method in the with_structured_output function.
@oracle-contributor-agreement oracle-contributor-agreement bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label Oct 29, 2025
method: Literal[
"function_calling", "json_schema", "json_mode"
] = "function_calling",
] = "json_schema",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you double check if the langchain openai is still using function_calling as default? Changing default might introduce unexpected behavior for customer.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. langchain openai is still using function_calling as default. See reference:
    https://github.com/langchain-ai/langchain/blob/master/libs/partners/openai/langchain_openai/chat_models/base.py

  2. The note in our repo said:
    If "json_schema" then it allows the user to pass a json schema (or pydantic) to the model for structured output. This is the default method.

That is why I change the default method to json_schema. I thought using function_calling as default was a typo.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's probably a mistake in the docstring. Let's not update the default and move that "This is the default method." to the correct place.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of course. I have changed the default back to function calling. Also commented in README that please use json_schema if the default does not work.

Remove import and define an alias for the class in the init
Changed the default method for structured output in ChatOCIGenAI from 'json_schema' to 'function_calling'. Updated documentation to clarify the default and suggest alternatives if it fails.
Clarify with_structured_output methods in README
@YouNeedCryDear YouNeedCryDear merged commit 8b18374 into oracle:main Oct 30, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

OCA Verified All contributors have signed the Oracle Contributor Agreement.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants