openai.Deployment.Create - deployment name setting not working

[nanditamohan]

Deployment name not setting with python api

I need to create a model deployment in my Azure Open AI workspace, specifically using openai.Deployment.create() python API. There is no clear spec on what needs to be passed in here, so I have not been able to get deployment name to set correctly.

I've tried setting the following params for setting it: name, deployment_name, deployment_id, openai_id, deployment, id

None seem to work - all set a generic default - I'm not sure if this functionality is broken or there is another param name.

openai.Deployment.create(model=completion_params["model_name"], name=completion_params["deployment_id"], scale_settings=scale_settings)

Currently, its been setting a random default Id:

Please help here with guidance.

To Reproduce

Run:

scale_settings = {
                "scale_type": "standard",
            }
openai.Deployment.create(model="text-chat-davinci-002", name="test-deployment-name", scale_settings=scale_settings)

Check AOAI workspace to see deployment name , and its not set to what I want

Code snippets

No response

OS

Windows

Python version

3.8.16

Library version

openai 0.27.6

[nanditamohan]

@hallacy any guidance here, is there a different param to set deployment name other than what I've tried or is this a bug?

[kristapratico]

@nanditamohan I don't believe the Deployment.create() API supports setting a deployment name. You can use the azure-mgmt-cognitiveservices library to create a model deployment and give it a custom name using the deployment_name keyword argument:

from azure.mgmt.cognitiveservices import CognitiveServicesManagementClient
client = CognitiveServicesManagementClient(credential=<credential>, subscription_id=<subscription_id>)

deployment = client.deployments.begin_create_or_update(
    resource_group_name="<resource_group_name>",
    account_name="<account_name>",
    deployment_name="<deployment_name>",
    deployment={
        "properties": {
          "model": {
            "format": "OpenAI",
            "name": "<model name>",
            "version": "<model version>"
          },
          "scaleSettings": {
            "scaleType": "Standard"
          }
        }
    }
)
result = deployment.result()
print(result)

[nanditamohan]

@kristapratico thanks for the response! Do you know if the cogservices library has a way to do this with the openai endpoint + key, rather than the account name and subscription info, similar to the openai API? I am working in a circumstance where that is what we have available. Thanks for the help

[kristapratico]

@nanditamohan can you describe the environment you're working in?

[nanditamohan]

Definitely @kristapratico, so we will have the customer's AzureML workspace credentials (subId, resourcegroup, workspace name for AzureML, but not for their cognitive services AOAI resource), and we will have their AzureOpenAI base, key, type and version - and will be validating and creating deployment for them from that information. I was able to successfully call openai.Deployment.retrieve to get a deployment by ID with only that information, with the following:

openai.api_type = <openai_api_type>
openai.api_version = <openai_api_version>
openai.api_base = <openai_api_base>
openai.api_key = <openai_api_key>
response = openai.Deployment.retrieve(**id**=model_params["deployment_id"])  

however, I can't create a deployment in this same way with this same id field. I am wondering if support could be added for something like this, taking in a similar id field to name the deployment

[nanditamohan]

@kristapratico let me know if you know anything about how I could approach this. Thanks for the help!

[kristapratico]

This conversation was taken offline, but want to post this here in case anyone else runs into the same issue:

Moving forward (Azure API versions 2023-05-15 and newer), the deployments APIs are deprecated in the openai library. They will remain accessible in earlier versions (like 2023-03-15-preview). For all code managing deployments, it is recommended to use the azure-mgmt-cognitiveservices client library. This library has additional options for working with deployments, like setting a deployment name or specific model version, and is the place where new features for deployments will be added.

Here are some samples to assist in migrating the code:

Create a deployment
Retrieve a deployment
List deployments
Delete a deployment

[iamramengirl]

Hi @kristapratico is there a way to set the rate limit (TPM) of the deployment programmatically through the azure-mgmt-cognitiveservices client library? Thank you!

[kristapratico]

@iamramengirl apologize for the late response. You can set the desired TPM limits with the latest version of the API, 2023-05-01. The sku.capacity property is used to set it.

Docs: https://learn.microsoft.com/en-us/rest/api/cognitiveservices/accountmanagement/deployments/create-or-update?tabs=HTTP

[Alisultani1]

Slowly

…

On Tue, 18 Jul 2023, 22:03 Krista Pratico ***@***.***> wrote: @iamramengirl <https://github.com/iamramengirl> apologize for the late response. You can set the desired TPM limits with the latest version of the API, 2023-05-01. The sku.capacity property is used to set it. Docs: https://learn.microsoft.com/en-us/rest/api/cognitiveservices/accountmanagement/deployments/create-or-update?tabs=HTTP — Reply to this email directly, view it on GitHub <#457 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A6AGSLCHZCDNDEGJPCL3LILXQ3FYLANCNFSM6AAAAAAYH6LSOI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[iamramengirl]

@iamramengirl apologize for the late response. You can set the desired TPM limits with the latest version of the API, 2023-05-01. The sku.capacity property is used to set it.

Docs: https://learn.microsoft.com/en-us/rest/api/cognitiveservices/accountmanagement/deployments/create-or-update?tabs=HTTP

Thank you for this! Really helpful!

[rattrayalex]

This library no longer has support for creating deployments. It may come back in the future.