✨ feat: support chat completion call at client side #2041

cy948 · 2024-04-14T15:22:54Z

💻 变更类型 | Change Type

🔀 变更说明 | Description of Change

detail:

♻️ src/libs/agent-runtime/AgentRuntime.ts：从 src/app/api/chat/agentRuntime.ts 中将 AgentRuntime 代码抽离，去掉服务端逻辑后得到该 lib； ❇️ chat 方法现在接受两个参数：payload:ChatStreamPayload、options?:ChatCompetitionOptions;
🧪 src/libs/agent-runtime/AgentRuntime.test.ts：为上述模块的测试文件，主要测试 AgentRuntime 类能否初始化及正常入参；
♻️ ✨ src/app/api/chat/agentRuntime.ts：新增方法 getLlmOptionsFromPayload 用于处理服务端参数、新增方法 initializeWithUserPayload 用于调用 src/libs/agent-runtime/AgentRuntime.ts 中的 AgentRuntime，并调用 getLlmOptionsFromPayload 进行参数处理；❇️L485: 新增了方法 createTraceOptions 用于构建 trace 设置。
🧪 src/app/api/chat/agentRuntime.test.ts：为上述新增方法 initializeWithUserPayload, getLlmOptionsFromPayload 补充单元测试，主要测试：
- ① initializeWithUserPayload 在空 payload 下能否正常使用环境变量填补必要的参数，完成 AgentRuntime instance 初始化；
- ② initializeWithUserPayload 在非空 payload 下能否正常传参，完成 AgentRuntime instance 初始化；
♻️ src/app/api/chat/[provider]/route.ts：使用 src/app/api/chat/agentRuntime.ts: initializeWithUserPayload 创建服务端 AgentRuntime instance；❇️根据是否启用trace条件调用chat方法。
🧪 src/app/api/chat/[provider]/route.test.ts：测试对象变更，src/app/api/chat/agentRuntime.ts -> src/libs/agent-runtime/AgentRuntime.ts ；❇️L113:115，新 route.ts 中若用户不开启 trace，chat 方法中没有第二个参数，故移除了对第二个参数的expect；
✨ src/services/chat.ts：新增方法 initializeWithClientStore 用于创建客户端 AgentRuntime instance；❇️移除了 L:178 中 fetchOnClient 方法的 trace 头构建；
🧪 src/services/__tests__/chat.test.ts：新增针对上述方法 initializeWithClientStore 的测试；❗ZhiPu AI 在初始化中，generateApiToken 方法报错，只能强制mock该方法结果以通过测试，日后待排查。
🌐 src/locales/default/setting.ts：新增 fetchOnClient 的翻译，❗ i18n 待 review 后生成；

📝 补充信息 | Additional Information

src/services/chat.ts:L271：使用用户 llm 设置决定是否使用 client 请求；
src/services/chat.ts:L85：目前仅在客户端设置解除 openai 的 sdk 的 browser 端限制，其他 sdk 可能也存在 browser 端限制，目前依靠其他 provider contributor 测试后进行调整；

preview: on Vercel
refs: [RFC] 024 - 浏览器端请求方式改造 #1257

vercel · 2024-04-14T15:22:57Z

Someone is attempting to deploy a commit to the LobeHub Team on Vercel.

A member of the Team first needs to authorize it.

lobehubbot · 2024-04-14T15:23:07Z

👍 @cy948

Thank you for raising your pull request and contributing to our Community
Please make sure you have followed our contributing guidelines. We will review it as soon as possible.
If you encounter any problems, please feel free to connect with us.
非常感谢您提出拉取请求并为我们的社区做出贡献，请确保您已经遵循了我们的贡献指南，我们会尽快审查它。
如果您遇到任何问题，请随时与我们联系。

arvinxx

Agent Runtime 里应该还是有一些 server 端的逻辑，估计还是需要一些改造。

从 server 取环境变量的逻辑需要抽离出来；
trace 的部分可能不需要有，或者要改成通过 api 去调用；

src/libs/agent-runtime/utils/openaiCompatibleFactory/index.ts

src/services/chat.ts

codecov · 2024-04-16T07:55:05Z

Codecov Report

Attention: Patch coverage is 94.50355% with 31 lines in your changes are missing coverage. Please review.

Project coverage is 92.81%. Comparing base (b3a7329) to head (8234092).

Files	Patch %	Lines
src/services/chat.ts	85.81%	21 Missing ⚠️
src/app/api/chat/[provider]/route.ts	38.46%	8 Missing ⚠️
src/libs/agent-runtime/AgentRuntime.ts	98.95%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2041      +/-   ##
==========================================
+ Coverage   92.77%   92.81%   +0.04%     
==========================================
  Files         265      266       +1     
  Lines       15086    15361     +275     
  Branches     1062     1783     +721     
==========================================
+ Hits        13996    14258     +262     
- Misses       1090     1103      +13

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

src/app/api/chat/[provider]/route.test.ts

src/app/api/chat/[provider]/route.ts

src/services/__tests__/chat.test.ts

src/app/api/chat/[provider]/route.test.ts

src/app/api/chat/agentRuntime.ts

src/libs/agent-runtime/AgentRuntime.ts

src/app/api/chat/models/[provider]/route.ts

src/libs/agent-runtime/AgentRuntime.ts

src/app/api/chat/[provider]/route.test.ts

src/app/api/chat/agentRuntime.ts

endpoint only - 🐛 fix: this feature only enable when user set both apikey and endpoint in one specific llm setting

* ♻️ refactor: remove deprecate class AgentRuntime * ♻️ refactor: fix circular dependence from AgentRuntime * ✅ test: pass tests

vercel · 2024-04-20T04:09:48Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
lobe-chat-community	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Apr 20, 2024 4:15am

lobehubbot · 2024-04-20T09:56:20Z

❤️ Great PR @cy948 ❤️

The growth of project is inseparable from user feedback and contribution, thanks for your contribution! If you are interesting with the lobehub developer community, please join our discord and then dm @arvinxx or @canisminor1990. They will invite you to our private developer channel. We are talking about the lobe-chat development or sharing ai newsletter around the world.
项目的成长离不开用户反馈和贡献，感谢您的贡献! 如果您对 LobeHub 开发者社区感兴趣，请加入我们的 discord，然后私信 @arvinxx 或 @canisminor1990。他们会邀请您加入我们的私密开发者频道。我们将会讨论关于 Lobe Chat 的开发，分享和讨论全球范围内的 AI 消息。

## [Version 0.148.0](v0.147.22...v0.148.0) <sup>Released on **2024-04-20**</sup> #### ✨ Features - **misc**: Support chat completion call at client side. <br/> <details> <summary><kbd>Improvements and Fixes</kbd></summary> #### What's improved * **misc**: Support chat completion call at client side, closes [#2041](#2041) ([9f5858f](9f5858f)) </details> <div align="right"> [![](https://img.shields.io/badge/-BACK_TO_TOP-151515?style=flat-square)](#readme-top) </div>

lobehubbot · 2024-04-20T09:59:55Z

🎉 This PR is included in version 0.148.0 🎉

The release is available on:

Your semantic-release bot 📦🚀

## [Version 1.28.0](v1.27.20...v1.28.0) <sup>Released on **2024-04-20**</sup> #### ✨ Features - **misc**: Support chat completion call at client side. <br/> <details> <summary><kbd>Improvements and Fixes</kbd></summary> #### What's improved * **misc**: Support chat completion call at client side, closes [lobehub#2041](https://github.com/bentwnghk/lobe-chat/issues/2041) ([9f5858f](9f5858f)) </details> <div align="right"> [![](https://img.shields.io/badge/-BACK_TO_TOP-151515?style=flat-square)](#readme-top) </div>

* ✨ feat: support chat completion call at client side * 🐛 fix: disable agent-runtime in browser feature when user set endpoint only - 🐛 fix: this feature only enable when user set both apikey and endpoint in one specific llm setting * ♻️ refactor: refactor agentruntime * ♻️ refactor: add fetchOnClient * 🚨 chore: eslint Circular Dependencies issue * 💄 feat: Added UI to control fetchOnClient * ✅ test: Passed chat provider test * ♻️ refactor: remove redundant export * ✅ test: add for agentRuntime * 💄 style: remove tag for llm config * ✅ test: add for agentRuntime init * ♻️ refactor: fetchOnClient * ✅ test: add for clinet agentRuntime init * ♻️ refactor: for write tests easily * ✅ test: for client AgentRuntime * 🌐 i18n: fetch on client * ✅ test: agentruntime * ♻️ refactor: don't trace on client * 📝 docs: update annotation for new methods * 🐛fix: error not catch at client * ♻️ refactor: extract trace at chat() * ✅ test: for new chat method * ♻️ refactor: -lines make stats better * ♻️ refactor: remove deprecate * ♻️ refactor: remove deprecate class AgentRuntime * ♻️ refactor: fix circular dependence from AgentRuntime * ✅ test: pass tests * ♻️ refactor: reformat import urls * ♻️ refactor: rename initAgentRuntimeWithUserPayload * 💄 style: improve config * 🌐 style: add i18n * 💄 style: client 默认显示 ollama

## [Version 0.148.0](lobehub/lobe-chat@v0.147.22...v0.148.0) <sup>Released on **2024-04-20**</sup> #### ✨ Features - **misc**: Support chat completion call at client side. <br/> <details> <summary><kbd>Improvements and Fixes</kbd></summary> #### What's improved * **misc**: Support chat completion call at client side, closes [lobehub#2041](lobehub#2041) ([9f5858f](lobehub@9f5858f)) </details> <div align="right"> [![](https://img.shields.io/badge/-BACK_TO_TOP-151515?style=flat-square)](#readme-top) </div>

jinjianming · 2024-07-11T12:19:53Z

docker 部署如何开启默认使用客户端模式呢

lobehubbot · 2024-07-11T12:20:06Z

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

How to enable docker deployment to use client mode by default

cy948 · 2024-07-11T15:36:40Z

docker 部署如何开启默认使用客户端模式呢

目前“是否默认开启客户端模式”功能由社区统一配置，暂时不能由环境变量进行配置。

lobehubbot · 2024-07-11T15:36:51Z

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

How to enable the default client mode for docker deployment

Currently, the "whether to enable client mode by default" function is uniformly configured by the community and cannot be configured by environment variables for the time being.

* ✨ feat: support chat completion call at client side * 🐛 fix: disable agent-runtime in browser feature when user set endpoint only - 🐛 fix: this feature only enable when user set both apikey and endpoint in one specific llm setting * ♻️ refactor: refactor agentruntime * ♻️ refactor: add fetchOnClient * 🚨 chore: eslint Circular Dependencies issue * 💄 feat: Added UI to control fetchOnClient * ✅ test: Passed chat provider test * ♻️ refactor: remove redundant export * ✅ test: add for agentRuntime * 💄 style: remove tag for llm config * ✅ test: add for agentRuntime init * ♻️ refactor: fetchOnClient * ✅ test: add for clinet agentRuntime init * ♻️ refactor: for write tests easily * ✅ test: for client AgentRuntime * 🌐 i18n: fetch on client * ✅ test: agentruntime * ♻️ refactor: don't trace on client * 📝 docs: update annotation for new methods * 🐛fix: error not catch at client * ♻️ refactor: extract trace at chat() * ✅ test: for new chat method * ♻️ refactor: -lines make stats better * ♻️ refactor: remove deprecate * ♻️ refactor: remove deprecate class AgentRuntime * ♻️ refactor: fix circular dependence from AgentRuntime * ✅ test: pass tests * ♻️ refactor: reformat import urls * ♻️ refactor: rename initAgentRuntimeWithUserPayload * 💄 style: improve config * 🌐 style: add i18n * 💄 style: client 默认显示 ollama

## [Version 0.148.0](lobehub/lobe-chat@v0.147.22...v0.148.0) <sup>Released on **2024-04-20**</sup> #### ✨ Features - **misc**: Support chat completion call at client side. <br/> <details> <summary><kbd>Improvements and Fixes</kbd></summary> #### What's improved * **misc**: Support chat completion call at client side, closes [lobehub#2041](lobehub#2041) ([9f5858f](lobehub@9f5858f)) </details> <div align="right"> [![](https://img.shields.io/badge/-BACK_TO_TOP-151515?style=flat-square)](#readme-top) </div>

arvinxx reviewed Apr 14, 2024

View reviewed changes

src/libs/agent-runtime/utils/openaiCompatibleFactory/index.ts Outdated Show resolved Hide resolved

src/services/chat.ts Outdated Show resolved Hide resolved

src/services/chat.ts Outdated Show resolved Hide resolved

src/services/chat.ts Outdated Show resolved Hide resolved

cy948 force-pushed the feat/agent-runtime-in-browser branch from c3c8148 to f57c7e1 Compare April 15, 2024 14:40

cy948 commented Apr 18, 2024

View reviewed changes

src/app/api/chat/[provider]/route.test.ts Show resolved Hide resolved

cy948 commented Apr 18, 2024

View reviewed changes

src/app/api/chat/[provider]/route.ts Show resolved Hide resolved

cy948 commented Apr 18, 2024

View reviewed changes

src/services/__tests__/chat.test.ts Show resolved Hide resolved

arvinxx reviewed Apr 19, 2024

View reviewed changes

cy948 commented Apr 19, 2024

View reviewed changes

src/app/api/chat/models/[provider]/route.ts Outdated Show resolved Hide resolved

cy948 commented Apr 19, 2024

View reviewed changes

src/libs/agent-runtime/AgentRuntime.ts Outdated Show resolved Hide resolved

lobehub deleted a comment from lobehubbot Apr 19, 2024

arvinxx reviewed Apr 19, 2024

View reviewed changes

src/app/api/chat/[provider]/route.test.ts Outdated Show resolved Hide resolved

src/app/api/chat/agentRuntime.ts Outdated Show resolved Hide resolved

cy948 and others added 17 commits April 20, 2024 01:09

✨ feat: support chat completion call at client side

2488081

🐛 fix: disable agent-runtime in browser feature when user set

bf671b6

endpoint only - 🐛 fix: this feature only enable when user set both apikey and endpoint in one specific llm setting

♻️ refactor: refactor agentruntime

1b458c7

♻️ refactor: add fetchOnClient

ae5cf6b

🚨 chore: eslint Circular Dependencies issue

00afc94

💄 feat: Added UI to control fetchOnClient

58645ab

✅ test: Passed chat provider test

37b0334

♻️ refactor: remove redundant export

60b396a

✅ test: add for agentRuntime

9260500

💄 style: remove tag for llm config

048e5f7

✅ test: add for agentRuntime init

5449a0b

♻️ refactor: fetchOnClient

5771727

✅ test: add for clinet agentRuntime init

bd45b57

♻️ refactor: for write tests easily

33600d3

✅ test: for client AgentRuntime

1376b9e

🌐 i18n: fetch on client

f72b8ce

✅ test: agentruntime

352e96b

cy948 added 6 commits April 20, 2024 01:09

♻️ refactor: extract trace at chat()

eb45b61

✅ test: for new chat method

c640e70

♻️ refactor: -lines make stats better

c0a6121

♻️ refactor: remove deprecate

1a95e3f

* ♻️ refactor: remove deprecate class AgentRuntime * ♻️ refactor: fix circular dependence from AgentRuntime * ✅ test: pass tests

♻️ refactor: reformat import urls

fe02ff5

♻️ refactor: rename initAgentRuntimeWithUserPayload

ec5be8c

arvinxx force-pushed the feat/agent-runtime-in-browser branch from fbc2741 to ec5be8c Compare April 19, 2024 17:09

arvinxx added 3 commits April 20, 2024 11:59

💄 style: improve config

381eb02

🌐 style: add i18n

5aa559c

💄 style: client 默认显示 ollama

8234092

vercel bot deployed to Preview April 20, 2024 04:15 View deployment

arvinxx merged commit 9f5858f into lobehub:main Apr 20, 2024
6 checks passed

arvinxx mentioned this pull request Apr 20, 2024

✨ feat: support minimax as provider #2087

Merged

7 tasks

lobehubbot added the released label Apr 20, 2024

cy948 deleted the feat/agent-runtime-in-browser branch April 20, 2024 15:51

arvinxx mentioned this pull request Apr 25, 2024

🐛 fix: fix chat client request not support abort #2193

Merged

7 tasks

jinjianming mentioned this pull request Jul 11, 2024

[Request] docker env设置开启客户端聊天 #3198

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

✨ feat: support chat completion call at client side #2041

✨ feat: support chat completion call at client side #2041

cy948 commented Apr 14, 2024 •

edited by arvinxx

Loading

vercel bot commented Apr 14, 2024

lobehubbot commented Apr 14, 2024

arvinxx left a comment

codecov bot commented Apr 16, 2024 •

edited

Loading

vercel bot commented Apr 20, 2024 •

edited

Loading

lobehubbot commented Apr 20, 2024

lobehubbot commented Apr 20, 2024

jinjianming commented Jul 11, 2024

lobehubbot commented Jul 11, 2024

cy948 commented Jul 11, 2024

lobehubbot commented Jul 11, 2024

✨ feat: support chat completion call at client side #2041

✨ feat: support chat completion call at client side #2041

Conversation

cy948 commented Apr 14, 2024 • edited by arvinxx Loading

💻 变更类型 | Change Type

🔀 变更说明 | Description of Change

📝 补充信息 | Additional Information

vercel bot commented Apr 14, 2024

lobehubbot commented Apr 14, 2024

arvinxx left a comment

Choose a reason for hiding this comment

codecov bot commented Apr 16, 2024 • edited Loading

Codecov Report

vercel bot commented Apr 20, 2024 • edited Loading

lobehubbot commented Apr 20, 2024

lobehubbot commented Apr 20, 2024

jinjianming commented Jul 11, 2024

lobehubbot commented Jul 11, 2024

cy948 commented Jul 11, 2024

lobehubbot commented Jul 11, 2024

cy948 commented Apr 14, 2024 •

edited by arvinxx

Loading

codecov bot commented Apr 16, 2024 •

edited

Loading

vercel bot commented Apr 20, 2024 •

edited

Loading