Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: close #935 add azure support #3206

Merged
merged 2 commits into from
Nov 9, 2023
Merged

feat: close #935 add azure support #3206

merged 2 commits into from
Nov 9, 2023

Conversation

Yidadaa
Copy link
Collaborator

@Yidadaa Yidadaa commented Nov 9, 2023

No description provided.

@Yidadaa Yidadaa merged commit 1141cd2 into main Nov 9, 2023
1 check passed
@Yidadaa Yidadaa deleted the azure branch November 9, 2023 18:55
H0llyW00dzZ pushed a commit to H0llyW00dzZ/ChatGPT-Next-Web that referenced this pull request Nov 9, 2023
H0llyW00dzZ added a commit to H0llyW00dzZ/ChatGPT-Next-Web that referenced this pull request Nov 9, 2023
* Merge pull request ChatGPTNextWeb#3206 from Yidadaa/azure

* doc: update azure env vars

* Update README.md

* Update README.md

* Fix Local Language

[+] fix(locales): fix translations for Token, AccessCode, and Endpoint in both cn.ts and en.ts files

* Fix Access Control

[+] fix(sync.ts): fix accessControl token property name to openaiApiKey

---------

Co-authored-by: Yidadaa <yidadaa@qq.com>
DirkSchlossmacher added a commit to DirkSchlossmacher/ChatGPT-Next-Web-AdEx that referenced this pull request Nov 10, 2023
commit e6b72ac
Author: Yifei Zhang <yidadaa@qq.com>
Date:   Fri Nov 10 02:59:47 2023 +0800

    Update README.md

commit 8032e6d
Author: Yifei Zhang <yidadaa@qq.com>
Date:   Fri Nov 10 02:59:30 2023 +0800

    Update README.md

commit c7e0a6f
Author: Yidadaa <yidadaa@qq.com>
Date:   Fri Nov 10 02:57:40 2023 +0800

    doc: update azure env vars

commit 1141cd2
Merge: fd2f441 c9dd953
Author: Yifei Zhang <yidadaa@qq.com>
Date:   Fri Nov 10 02:55:12 2023 +0800

    Merge pull request ChatGPTNextWeb#3206 from Yidadaa/azure

commit c9dd953
Author: Yidadaa <yidadaa@qq.com>
Date:   Fri Nov 10 02:50:50 2023 +0800

    fixup

commit b7ffca0
Author: Yidadaa <yidadaa@qq.com>
Date:   Fri Nov 10 02:43:30 2023 +0800

    feat: close ChatGPTNextWeb#935 add azure support

commit fd2f441
Author: Yifei Zhang <yidadaa@qq.com>
Date:   Thu Nov 9 20:45:25 2023 +0800

    feat: wont send max_tokens
@Vigilans
Copy link

目前chatgpt-next-web似乎只能支持指定一个azure endpoint(而且连deployment都指定了)。不过Azure OpenAI是一个多Endpoint、多Deployment的设计。参考在 binary-husky/gpt_academic#1173 (comment) 中的评论:

除了同 endpoint 多个模型以外,还有多 endpoint 的情况:GPT-4 模型不是每个地区随时可选的,目前貌似只有 US East 和 Sweden 有,而 Japan East 地区到国内的连通性更好(联通 ping 只有 40-50ms),但目前只有 GPT-3.5 可选。因此作为权衡方案可以让 3.5/3.5-16k 的请求发给 JP,4 的请求发给 SE。

azure-openai-proxy 就支持这样的设置:

api_base: "/v1"
deployment_config:
  - deployment_name: "xxx"
    model_name: "text-davinci-003"
    endpoint: "https://xxx-east-us.openai.azure.com/"
    api_key: "11111111111"
    api_version: "2023-03-15-preview"
  - deployment_name: "yyy"
    model_name: "gpt-3.5-turbo"
    endpoint: "https://yyy.openai.azure.com/"
    api_key: "11111111111"
    api_version: "2023-03-15-preview"
  - deployment_name: "zzzz"
    model_name: "text-embedding-ada-002"
    endpoint: "https://zzzz.openai.azure.com/"
    api_key: "11111111111"
    api_version: "2023-03-15-preview"

所以个人感觉相比于指定全局的服务提供商,为每个模型指定其使用的提供商要更好。这样的话可以做到:

  • gpt-3.5-turbo, gpt-4-32k 这样的模型走azure
  • gpt-4-1106-preivew这样的新模型仍然fallback到openai
  • 可以在同一个app设置下同时支持llama,claude等模型。切到对应的模型后会用对应的提供商。
  • 可以支持api2d这样的与azure类似的openai模型提供商。模型名字可以取api2d-gpt-3.5-turbo这样的,从而与gpt-3.5-turbo与azure-gpt-3.5-turbo共存。

@Vigilans
Copy link

Vigilans commented Nov 13, 2023

目前的Azure设置里面,URL会一直指定到deployment。但是,azure里实际上是一个deployment对应一个模型的:
image

对应gpt-3.5与gpt-4,需要使用两个不同的deployment。


因此,在chatgpt-next-web里选择任何一个模型,都会使用同一个deployment,这导致了另一个问题::
image
image
chatgpt-next-web使用了gpt-3.5-turbo的模型,向一个gpt-4的deployment发了请求。

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


In the current Azure setup, the URL is specified all the way to the deployment. However, in Azure, one deployment actually corresponds to one model:
Uploading image.png…
Corresponding to gpt-3.5 and gpt-4, two different deployments need to be used.


Therefore, any model selected in chatgpt-next-web will use the same deployment, which leads to another problem:
image
image
I used the gpt-3.5-turbo model and sent a request to a gpt-4 deployment.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants