-
Notifications
You must be signed in to change notification settings - Fork 59
Closed
Labels
Description
Is your feature request related to a problem?
Current only scheduler node can hold chatbot, we should let every node can hold a local chatbot.
Describe the Solution you'd like
- Reuse current chatbot frontend
- Build a new simple http server for node, http server forward request(chat completion and cluster status api) to scheduler lattica rpc handler @sibianl
- scheduler rpc handler forward request to scheduler http api @gufengc
- launch frontend and http server on node(do not launch it on scheduler) @sibianl
Alternatives Considered (Optional)
No response
Additional Context (Optional)
No response