I wanted to report a small bug when using the function chat_structured function in combination with type_from_schema with a deeply nested JSON schema. This is related to issue #529.
It seems that there is an openAI schema limitation that does not allow for object nesting deeper than 5 levels. If you use more levels than that, you get an error like this:
ℹ : Invalid schema for response_format 'structured_data': 6 levels of nesting exceeds limit of 5.
However, what I found is that the current implementation of the chat_structured function contains the following lines:
needs_wrapper <- S7_inherits(private$provider, ProviderOpenAI)
type <- wrap_type_if_needed(type, needs_wrapper)
The result is that the passed schema is wrapped one more time as an object. This results in one more additional nesting level, that should not be there.
I confirmed that if I manually set needs_wrapper <- FALSE, then the structured_chat call works fine.
Here is a reprex that you can use as a test for this behavior.
chat <- chat_openai(model = "o3", system_prompt = "You are a baker")
recipe <- "Preheat the oven to 350 degrees F (175 degrees C)."
result <- chat$chat_structured(recipe,
type = ellmer::type_from_schema('{
"title": "Cookie Schema",
"description": "Test",
"type": "object",
"required":["recipes"],
"additionalProperties": false,
"properties":{
"recipes": {
"type": "array",
"description": "List of detailed information on each of recipes",
"items":{
"type": "object",
"description": "Detailed information about a specific recipe.",
"required": [ "equipment"],
"additionalProperties": false,
"properties": {
"equipment": {
"type": "array",
"description": "Detailed list of all equipment required.",
"items": {
"type": "object",
"properties": {
"details": {
"type":"object",
"description": "Specifications and operating conditions.",
"required": ["specification"],
"additionalProperties": false,
"properties": {
"specification": {
"type": "object",
"description": "Equipment specifications like temperature range",
"required": ["temperature"],
"additionalProperties": false,
"properties": {
"temperature": {"type": "string", "description": "e.g., 100F, 500F"}
}
}
}
}
},
"additionalProperties": false,
"required": ["details"]
}
}
}
}
}
}
}
'))
This is the expected result for the example above. I see this when setting the needs_wrapper <- FALSE inside the chat_structured function.
$recipes
$recipes[[1]]
$recipes[[1]]$equipment
$recipes[[1]]$equipment[[1]]
$recipes[[1]]$equipment[[1]]$details
$recipes[[1]]$equipment[[1]]$details$specification
$recipes[[1]]$equipment[[1]]$details$specification$temperature
[1] "350F"
I wanted to report a small bug when using the function
chat_structuredfunction in combination withtype_from_schemawith a deeply nested JSON schema. This is related to issue #529.It seems that there is an openAI schema limitation that does not allow for object nesting deeper than 5 levels. If you use more levels than that, you get an error like this:
However, what I found is that the current implementation of the
chat_structuredfunction contains the following lines:The result is that the passed schema is wrapped one more time as an object. This results in one more additional nesting level, that should not be there.
I confirmed that if I manually set
needs_wrapper <- FALSE, then thestructured_chatcall works fine.Here is a reprex that you can use as a test for this behavior.
This is the expected result for the example above. I see this when setting the
needs_wrapper <- FALSEinside thechat_structuredfunction.