Add gradio chatbot for openai webserver

[arkohut]

Follow the example of https://github.com/vllm-project/vllm/blob/main/examples/gradio_webserver.py I create a gradio webserver for openai api server.

Right now I find that OpenAI-compatible api server has more features than the default webserver and is much more popular. This script will create a chatbot to test the web server.

Here is a command example:

$ python chat.py -m '01ai/Yi-34B-Chat-4bits' --stop-token-ids 7

Running on local URL:  http://127.0.0.1:8001
Running on public URL: https://747eb488ee1e9155e4.gradio.live

This share link expires in 72 hours. For free permanent hosting and GPU upgrades, run `gradio deploy` from Terminal to deploy to Spaces (https://huggingface.co/spaces)

Here is the screenshot:

[Tostino]

I wanted to mention I've been using LibreChat for a little while now against my models hosted using vLLM OpenAI API. It's not as simple as a single file python app, but I found that it works really, really well as a ChatGPT replacement, which I can not say about things like Gradio interfaces.

Since other interfaces are finally becoming available that can integrate with the standard APIs provided, It may be smarter for the project to link to those compatible interfaces, with simple setup instructions. The more code you have in the repo that isn't the main purpose of the project, the harder it is to maintain high quality code and quickly deliver features.

[arkohut]

The more code you have in the repo that isn't the main purpose of the project, the harder it is to maintain high quality code and quickly deliver features.

You are right. I will not use a single file like this in my daily work. I just found that a small test file for the openai api is really convenient in the same repository. And there is already one file https://github.com/vllm-project/vllm/blob/main/examples/gradio_webserver.py for the api testing.

So feel free to close this PR if you think it is not following the main purpose of the project. And I really understand the reason.

[FlorianJoncour]

I personally agree with this PR, but why not in the examples directory or maybe create a endpoint or client directory as a mirror of entrypoint

[zhuohan123]

Can you move it to examples/gradio_openai_chatbot_webserver.py?

[arkohut]

Can you move it to examples/gradio_openai_chatbot_webserver.py?

It's done.

[arkohut]

Do I need to do any other update? @zhuohan123

[zhuohan123]

LGTM! Thank you for your contribution!