json_object response_format not behaving as expected on Azure gpt-3.5-turbo-0125

[milstan]

Confirm this is an issue with the Python library and not an underlying OpenAI API

• This is an issue with the Python library

Describe the bug

Using the OpenAI python library with an Azure OpenAI instance, I am trying ot generate a json response guaranteed to be in json format (as only including it in text promt sometimes yields inadequate results).

For a request with the following parameters:

'model': 'gpt-3.5-turbo-0125', 'response_format': {'type': 'json_object'}

I am getting the following error:

openai.BadRequestError: Error code: 400 - {'error': {'message': "Invalid parameter: 'response_format' of type 'json_object' is not supported with this model.", 'type': 'invalid_request_error', 'param': 'response_format', 'code': None}}

Yet, my understnading is that (according to the Azure documentation page):

• json mode is supported by the model gpt-35-turbo (0125)
• and the 2024-02-01 API version which I use, supports json_object response_format

To Reproduce

1. Set up an Azure OpenAI instance
2. from openai.lib.azure import AsyncAzureOpenAI

client = AsyncAzureOpenAI(
api_key=,
api_version="2024-02-01",
azure_endpoint=,
azure_deployment=
)

response = await client.chat.completions.create(
model= "gpt-3.5-turbo-0125",
messages=[
{"role": "user", "content": },
],
response_format={'type': 'json_object'}
)

print (response.choices[0].message.content)

Code snippets

No response

OS

masOS

Python version

Python 3.11.2

Library version

openai v1.30.4

[kristapratico]

@milstan I'm having trouble reproducing the issue. Can you tell me what value you're passing in to azure_deployment here? Any chance it's different than the one you're passing for model?

client = AsyncAzureOpenAI(
  api_key=,
  api_version="2024-02-01",
  azure_endpoint=,
  azure_deployment=
)

[milstan]

thanks @kristapratico

Indeed, it was different azure_deployment=gpt-35-turbo-16k, and model=gpt-3.5-turbo-0125. What is missleading is the error message saying that the 'json_object' is not supported with this model. While in fact the model specified in the request was ignored, and the one deployed in the deployment gpt-35-turbo-16k was used (Please note that Azure sometimes allows to specify custom deployment names that may be unrelated to the model beying deployed).

Managed to solve this issue by making a new deployment (gpt-35-turbo-1106) of the model gpt-3.5-turbo-1106 and now it works.

[kristapratico]

@milstan glad you got it working. When you pass azure_deployment into the client, then any model value you pass at the method-level will be ignored. You can choose to not use azure_deployment and just pass your deployments as model=deployment if that works better for you.