feature: make non azure implement support different endpoint

[xmkevin]

Description

Sometimes, we want to use non azure implement for other OpenAI compatible endpoint, check azure.com for azure implement seems more nature.

All SDK Contribution checklist:

• The pull request does not introduce [breaking changes]
• CHANGELOG is updated for new features, bug fixes or other significant changes.
• I have read the contribution guidelines.

General Guidelines and Best Practices

• Title of the pull request is clear and informative.
• There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

• Pull request includes test coverage for the included changes.

[github-actions]

Thank you for your contribution @xmkevin! We will review the pull request and get back to you soon.

[xmkevin]

@microsoft-github-policy-service agree [company="Sky AI"]

[Benzhaomin]

We have the exact same need. I was just about to push the same kind of PR but checked open PRs to see if anything happened since yesterday and there it is! I tested your PR, it fits our usecase as well and just works. 🚀

For reference, here is my code 466ace9

[xmkevin]

@ agree company="Sky AI"

[mssfang]

@xmkevin @Benzhaomin
Thank you for your interest and contributions to OpenAI SDK.

May I know what the possible other endpoints could be exist other than the Azure and Public OpenAI endpoint which always started with https://api.openai.com/v1?

Appreciate it If more details on the scenarios is provided.

[xmkevin]

@xmkevin @Benzhaomin Thank you for your interest and contributions to OpenAI SDK.

May I know what the possible other endpoints could be exist other than the Azure and Public OpenAI endpoint which always started with https://api.openai.com/v1?

Appreciate it If more details on the scenarios is provided.

In some case, I need to put an nginx before openai api to use my own domain name. I try to use proxy configuration, but it seems doesn't work.

[Benzhaomin]

@xmkevin @Benzhaomin Thank you for your interest and contributions to OpenAI SDK.

May I know what the possible other endpoints could be exist other than the Azure and Public OpenAI endpoint which always started with https://api.openai.com/v1?

Appreciate it If more details on the scenarios is provided.

We want to support a new hosted LLM provider that exposes an OpenAI-compatible API. We already have integration with both OpenAI and Azure OpenAI so just being able to customize the endpoint for this other service would be great.

The service in question is https://endpoints.ai.cloud.ovh.net and an example of their API is https://mistral-7b-instruct-v02.endpoints.kepler.ai.cloud.ovh.net/doc#post-/api/openai_compat/v1/chat/completions

[mssfang]

Thank you for the reply. Let me bring this topic up to team to see if we should do it. Will keep you updated.

[viswanathk]

@mssfang another point in support of this - few companies have some gateway in front of the public facing endpoint through which they control the configuration (org id, auth etc)

this sdk doesn't support those cases.

[mssfang]

Hi, @xmkevin @Benzhaomin @viswanathk

Unfortunately, we will not support non-OpenAI models in this SDK, azure-ai-openai as per our partnership with OpenAI.
The most major models are available as first-class Azure Host Endpoints, such as Llama2, Cohere and Mistra, via Azure MaaS (Azure AI Studio Model Catalog). They can also host most Hugging Face models or your own models in the future via Azure MaaP(AML) in the future. We will provide an ("OpenAI like") Model Inference API spec for completions to MaaS in the near future.

Worth to mention here that this SDK supports proxy feature in case someone not aware of it.

[xmkevin]

Hi, @xmkevin @Benzhaomin @viswanathk

Unfortunately, we will not support non-OpenAI models in this SDK, azure-ai-openai as per our partnership with OpenAI. The most major models are available as first-class Azure Host Endpoints, such as Llama2, Cohere and Mistra, via Azure MaaS (Azure AI Studio Model Catalog). They can also host most Hugging Face models or your own models in the future via Azure MaaP(AML) in the future. We will provide an ("OpenAI like") Model Inference API spec for completions to MaaS in the near future.

Worth to mention here that this SDK supports proxy feature in case someone not aware of it.

ok, waiting for the new feature.