Problem with gradio server on Windows

[Florian0077]

Hello everyone, I managed to merge the ALPACA 13B model with the DELTA VICUNA and everything works correctly in the CLI mode.
However, I have a problem with the gradio server. When I run the command: "python -m fastchat.serve.gradio_web_server"
I always get the error:
"Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it'"

I checked the firewall and everything seems ok, I changed the port, etc... nothing changed.
I tested several times another application under gradio and it works well. Do any of you have the same issue?

Thank's for your help

[zhisbug]

Hi @Florian0077

Please follow the steps here: https://github.com/lm-sys/FastChat#web-ui
You must follow the three steps:

1. launch a controller
2. launch a worker and connect to the controller
3. launch Gradio webserver and connect to the controller.

You cannot launch a server alone.

Also, the delta weights are supposed to be applied to the llama weights, not the alpaca weights.

[Florian0077]

Thank you for the answer !
I actually used the checkpoint llama 13b and not alpaca (mistake in my first message)
I carried out the following procedure.

1.Create a new environment
2.Install fschat
3.Install cuda for my GPU (11.3 in my case)
4.Run powershell and activate env.
5.launch a controller
6. Run new powershell and activate env.
7.launch a worker and connect to the controller
exemple:
python -m fastchat.serve.model_worker --model-path F:\python\machine_learning\FastChat_env\vicuna-13b

but I get several errors.

Where is the mistake ?

Thank's

[saitejamalyala]

I face a similar issue in Linux as well, When model worker is launched, it does register with controller, but gradio doesn't pickup list of models (response is OK, but json is empty)

[zhisbug]

Resolved.

[zxcvbn114514]

I face a similar issue in Linux as well, When model worker is launched, it does register with controller, but gradio doesn't pickup list of models (response is OK, but json is empty)

Have you solved this problem?I met it too.