-
Notifications
You must be signed in to change notification settings - Fork 2k
Description
Expected Behavior
Base on the OpenAI document, when using Structured Outputs, outputs will be produced in the same order as the ordering of keys in the json schema. Thus the BeanOutputConverter.getJsonSchema() should keep the key ordering of the Record class to enforce the key order, which would provide benefit of Chain of thought
Current Behavior
The order of key is not preserved, meaning that sometimes it will generate result first, then reasoning. For example:
record MathReasoning(@JsonProperty(required = true, value = "steps") Steps steps,
@JsonProperty(required = true, value = "final_answer") String finalAnswer) {
record Steps(@JsonProperty(required = true, value = "items") Items[] items) {
record Items(@JsonProperty(required = true, value = "explanation") String explanation,
@JsonProperty(required = true, value = "output") String output) {
}
}
}
might produce result
{
"final_answer": "x = -3.75",
"steps": {
"items": [
{
"explanation": "Start with the original equation: 8x + 7 = -23",
"output": "8x + 7 = -23"
},
{
"explanation": "Subtract 7 from both sides to isolate the term with x.",
"output": "8x = -23 - 7"
},
{
"explanation": "Calculate -23 - 7 which equals -30.",
"output": "8x = -30"
},
{
"explanation": "Divide both sides by 8 to solve for x.",
"output": "x = -30 / 8"
},
{
"explanation": "Simplify -30 / 8 to get the final answer.",
"output": "x = -3.75"
}
]
}
}
which is final_answer showing first.
Context
In order to have better inference performance, it should be steps show up first, then final_answer. Using string directly for jsonSchema doesn't have the issue, but BeanOutputConverter can't keep the original key order, making it only useful on cases where key order doesn't matter.