This repository has been archived by the owner on Jan 9, 2024. It is now read-only.

add VLLM support #63

Closed

vkehfdl1 opened this issue Jun 28, 2023 · 3 comments

Contributor

vkehfdl1 commented Jun 28, 2023 •

edited

Loading

I can run KuLLM and polyglot. But KoAlpaca is hard to run with VLLM

This was linked to pull requests Jul 7, 2023

VLLM integration #76

Closed

VLLM integration #77

Closed

Contributor Author

vkehfdl1 commented Jul 7, 2023

There is no langchain integration of VLLM yet

Contributor Author

vkehfdl1 commented Jul 7, 2023

KuLLM : "nlpai-lab/kullm-polyglot-12.8b-v2"
polyglot-ko : EleutherAI/polyglot-ko-12.8b

Contributor Author

vkehfdl1 commented Aug 15, 2023

close at #108
We support vLLM as openAI API server now.

vkehfdl1 closed this as completed

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.