Skip to content

SetModelResponseTool: schema validation failure terminates the tool flow instead of returning feedback #1430

Description

@BlueCatPro

🔴 Required Information

Describe the Bug

SetModelResponseTool currently propagates output-schema validation failures as tool execution errors.

In the current Java implementation, SchemaUtils.validateMapOnSchema(...) is executed inside Single.fromCallable(...). When validation throws an IllegalArgumentException, the Single terminates with an error instead of returning validation feedback as a tool response.

Under the default tool execution flow, no validation-feedback FunctionResponse is returned to the model. As a result, the model cannot correct the arguments and call set_model_response again.

ADK Python now returns structured-output validation failures to the model as tool feedback and records successfully validated output separately. This appears to be a Java/Python parity gap.

There is also a related finalization concern: OutputSchema.getStructuredModelResponse(...) currently promotes a function response named set_model_response to the final structured response. Validation feedback must therefore be distinguishable from successfully validated output.

Steps to Reproduce

Create an output schema containing a required field:

Schema outputSchema =
    Schema.builder()
        .type("OBJECT")
        .properties(
            ImmutableMap.of(
                "field1",
                Schema.builder().type("STRING").build()))
        .required(ImmutableList.of("field1"))
        .build();

Create and invoke SetModelResponseTool without the required field:

SetModelResponseTool tool =
    new SetModelResponseTool(outputSchema);

Map<String, Object> invalidArgs = ImmutableMap.of();

tool.runAsync(invalidArgs, null).blockingGet();

The call terminates with:

java.lang.IllegalArgumentException:
Output args does not contain required field1

Expected Behavior

For parity with ADK Python, an output-schema validation failure from the internal set_model_response tool should be returned to the model as validation feedback:

LLM calls set_model_response
        ↓
schema validation fails
        ↓
validation feedback is returned to the model
        ↓
LLM corrects the arguments
        ↓
LLM calls set_model_response again
        ↓
validated result becomes the final response

An invalid set_model_response call must not become the final structured model response. Only successfully validated output should be finalized.

The exact Java implementation does not need to identify validation feedback solely through an "error" field, since a valid user-defined output schema could legitimately contain a field with that name.

Observed Behavior

SetModelResponseTool currently performs:

return Single.fromCallable(
    () -> {
      SchemaUtils.validateMapOnSchema(
          args, outputSchema, /* isInput= */ false);
      return args;
    });

The validation exception therefore becomes an RxJava error signal and propagates through the tool flow.

The existing upstream SetModelResponseToolTest explicitly verifies this behavior using assertThrows(...), confirming that exception propagation is the current expected Java behavior.

Additionally, OutputSchema.getStructuredModelResponse(...) currently identifies the final structured response using the set_model_response function name. A change that only converts the exception into a Python-style feedback map could therefore incorrectly promote that feedback to the final response unless validated output is tracked or distinguished separately.

Environment Details

  • ADK Java version: commit 2b87d65d9704a61ff4668b8c9482a79fef9fe0d4
  • Java: Microsoft OpenJDK 17.0.15
  • OS: Windows 11
  • Test framework: JUnit 4

Model Information

  • Model: N/A
  • The issue is reproducible deterministically at the tool level without a live model

🟡 Optional Information

Regression

No known Java regression.

This appears to be a behavioral parity gap following the newer structured-output validation recovery behavior in ADK Python.

Logs

IllegalArgumentException: Output args does not contain required field1
    at com.google.adk.SchemaUtils.validateMapOnSchema(SchemaUtils.java:124)
    at com.google.adk.tools.SetModelResponseTool.lambda$runAsync$0(
        SetModelResponseTool.java:63)

Additional Context

The corresponding Python behavior is present on adk-python main in:

google/adk-python@b5b27cb

The change originated from:

google/adk-python#6368

The Python implementation catches structured-output validation failures and returns validation feedback to the model. Successfully validated output is recorded separately and is the only result promoted to the final structured response.

Relevant Java files:

  • core/src/main/java/com/google/adk/tools/SetModelResponseTool.java
  • core/src/main/java/com/google/adk/flows/llmflows/OutputSchema.java
  • core/src/test/java/com/google/adk/tools/SetModelResponseToolTest.java

The ADK Java contribution guide states that ADK Java aims to align with ADK Python and leans on ADK Python as the source of truth for behavioral validation:

https://github.com/google/adk-java/blob/2b87d65d9704a61ff4668b8c9482a79fef9fe0d4/CONTRIBUTING.md#alignment-with-adk-python

Would maintainers agree that ADK Java should align with the current Python recovery behavior here?

How often has this issue occurred?

Always (100%) when SetModelResponseTool receives arguments that fail output-schema validation.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions