logprobs parameter support for Chat Completion API and GPT-4 model

[MRvra-MSFT]

Confirm this is an issue with the Python library and not an underlying OpenAI API

• This is an issue with the Python library

Describe the bug

May you please share a documentation regarding the correct usage of logprobs parameter in Chat Completion API as well as support for the following models.

1. gpt-4-0613
2. gpt-4-1106-Preview

Currently, when using the parameter with the above models the following errors are returned by the Chat Completion API. openai version is 1.7.2

gpt-4-06-13
BadRequestError: Error code: 400 - {'error': {'message': 'Unrecognized request argument supplied: logprobs', 'type': 'invalid_request_error', 'param': None, 'code': None}}

gpt-4-1106
BadRequestError: Error code: 400 - {'error': {'message': "This model does not support the 'logprobs' parameter.", 'type': 'invalid_request_error', 'param': 'logprobs', 'code': None}}

To Reproduce

Please see code snippet below.

Code snippets

from openai import AzureOpenAI
 
client = AzureOpenAI(
  azure_endpoint = 'API_ENDPOINT', 
  api_key='API_KEY',  
  api_version='2023-05-15',
  azure_deployment = 'gpt-4-1106'
)

response = client.chat.completions.create(
    model='gpt-4',
    messages=[
        {....}
    ],
    logprobs=True,
    top_logprobs=2
)
response

OS

Windows11

Python version

Python v3.11.2

Library version

openai v1.7.2

[kristapratico]

@iamramengirl logprobs and top_logprobs are not supported for chat completions by the Azure API quite yet. I expect support to come online soon, but don't have an ETA at the moment.

[MRvra-MSFT]

@kristapratico Thank you for the information!

[MRvra-MSFT]

Hello @kristapratico any rough ETA you may share as many customers are waiting for this feature? Thanks!

[kristapratico]

@iamramengirl unfortunately I cannot share details about timelines. I will update this thread once it is available. Thanks for understanding.

[MRvra-MSFT]

Thank you, @kristapratico!

[gopikrishnachaganti]

this feature will be very useful, waiting for the same. please inform when the feature will be available.

[RichDCs]

any update? thanks!

[kristapratico]

Not yet. Promise to come back and update the thread when it's available.

[rattrayalex]

This is a request for the Azure API, and not the Python SDK, so I've gone ahead and closed this issue – but you can keep an eye here to find out when this feature is available there.

[mrshu]

@kristapratico could you please still update this thread once it is available on Azure? I believe quite a few of us are subscribed to it because of that.

Thanks!

[rawsh]

For a while gpt4-turbo deployed on azure was working with logprobs=True, top_logprobs=5, now the same code is getting

openai.BadRequestError: Error code: 400 - {'error': {'message': "This model does not support the 'logprobs' parameter.", 'type': 'invalid_request_error', 'param': 'logprobs', 'code': None}}

[kristapratico]

@kristapratico could you please still update this thread once it is available on Azure? I believe quite a few of us are subscribed to it because of that.

Thanks!

Yes. AFAIK it is still not officially supported.

[Toubat]

For a while gpt4-turbo deployed on azure was working with logprobs=True, top_logprobs=5, now the same code is getting

openai.BadRequestError: Error code: 400 - {'error': {'message': "This model does not support the 'logprobs' parameter.", 'type': 'invalid_request_error', 'param': 'logprobs', 'code': None}}

@rawsh Ran into the same issue. Have you found a solution?