Google Gemini not supporting native structured output - instead always using tool calling method

[andrewdoro]

Checked other resources

• This is a bug, not a usage question. For questions, please use the LangChain Forum (https://forum.langchain.com/).
• I added a very descriptive title to this issue.
• I searched the LangChain.js documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain.js rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

// Define the output format selection schema
const OutputFormatSelectionSchema = z.object({
  query: z
    .string()
    .describe('The search query'),
});

const model = ChatGoogle({
          model: 'gemini-2.5-flash-lite',
          temperature: 0,
        });

  const structuredLlm = withStructuredOutput(model, OutputFormatSelectionSchema);

const result = await structuredLlm.invoke("Generate a search query")

Error Message and Stack Trace (if applicable)

No response

Description

With structuredOutput is defaulting to extract tool because of missing logic for Gemini native structured out.
This usually results in much worse results for JSON generation.

To get native json generation we would need to pass responseMimeType and responseSchema to the API request.
https://ai.google.dev/gemini-api/docs/structured-output#javascript

System Info

Node: v23.11.0
Mac OS X

langchain@0.3.30 | MIT | deps: 11 | versions: 325
Typescript bindings for langchain
https://github.com/langchain-ai/langchainjs/tree/main/langchain/

keywords: llm, ai, gpt3, chain, prompt, prompt engineering, chatgpt, machine learning, ml, openai, embeddings, vectorstores

dist
.tarball: https://registry.npmjs.org/langchain/-/langchain-0.3.30.tgz
.shasum: 9fc0467007009dc5e799d1f8fb9dd1137654e387
.integrity: sha512-UyVsfwHDpHbrnWrjWuhJHqi8Non+Zcsf2kdpDTqyJF8NXrHBOpjdHT5LvPuW9fnE7miDTWf5mLcrWAGZgcrznQ==
.unpackedSize: 2.9 MB

dependencies:
@langchain/openai: >=0.1.0 <0.7.0 openapi-types: ^12.1.3
@langchain/textsplitters: >=0.0.0 <0.2.0 p-retry: 4
js-tiktoken: ^1.0.12 uuid: ^10.0.0
js-yaml: ^4.1.0 yaml: ^2.2.1
jsonpointer: ^5.0.1 zod: ^3.25.32
langsmith: ^0.3.33

published a week ago by davidduong david@duong.cz

[afirstenberg]

I thought we had an open issue for this already, but I can't find it.
Hunter Lovell (@hntrl) - Please assign to me.

[andrewdoro]

thank you sm Allen Firstenberg (@afirstenberg)

also wanted to add more context, this is from ai sdk docs for the structured output
https://ai-sdk.dev/providers/ai-sdk-providers/google-generative-ai#schema-limitations

Schema Limitations
The Google Generative AI API uses a subset of the OpenAPI 3.0 schema, which does not support features such as unions. The errors that you get in this case look like this:

GenerateContentRequest.generation_config.response_schema.properties[occupation].type: must be specified

By default, structured outputs are enabled (and for tool calling they are required). You can disable structured outputs for object generation as a workaround:


The following Zod features are known to not work with Google Generative AI:

z.union
z.record

[StErMi]

From my understanding of the docs relative to the structured output

If the provider natively supports structured output for your model choice, it is functionally equivalent to write responseFormat: contactInfoSchema instead of responseFormat: toolStrategy(contactInfoSchema). In either case, if structured output is not supported, the agent will fall back to a tool calling strategy.

Langchain should automatically fall back to toolStrategy but it appears that it is not doing it at all throwing that error.

[lakshmanok]

Gemini does not allow you to built-in tools and Structured Outputs at the same time.
The best way out is to explicitly reformat the output with a second LLM call. Here's an example:

https://github.com/lakshmanok/lakblogs/blob/912aa40d79a3fcfaf127588681767955b06c3847/stockanalysis/3b_langchain_subagents.py#L138

[macrozone]

in the latest langchain version, providerStrategy does not even throw an error anymore, structuredResponse is simply silently null. toolStrategy seems to work with gemini, but has broken types.

[hntrl]

Apologies for the delay, this should be fixed around the time we launched support for native structured output from google