Websocket error

[nawalnew]

Ive got it to work locally on my M2 with the help of the issue before.

But I got stuck with the Websocket.
When I ask something, than it loads endless.

WebSocket connection to 'ws://localhost:8008/' failed: There was a bad response from the server.

Ive tried in chrome and Safari.

[kells1986]

I can't get to the chat page. After I click "Start a New Chat" it just hangs forever before I eventually get a 504 error. This is on my M1 machine, tried firefox and chrome

[nawalnew]

Sorry @kells1986 , I didnt run into this issue on my mac.
But look in your dev tools what is wrong there. Maybe the same underlying websocket issue.

[kells1986]

I'm seeing the same issue now too:

WebSocket connection to 'ws://localhost:8008/' failed: 
Direct websocket connection fallback. Check out https://vitejs.dev/config/server-options.html#server-hmr to remove the previous connection error.

More context: setupWebSocket @client.ts: 78

[nsarrazin]

I think this is due to the hot module reloading not going through nginx properly.

I fixed it on main with 58cf7d0 can you pull the latest main and tell me if it's fixed ? @kells1986 @nawalnew

[kells1986]

Working! Thanks, this is awesome!

[nsarrazin]

I'm gonna close this now, if you still have issues let me know and I will reopen it!

[dylancl]

Hi,

Hopefully you don't mind me posting in a closed issue, but I'm still facing the same issue on the latest commit on main running on a M1 MacBook. I can see the request successfully reaches the API, but it's infinitely stuck on loading (and still showing the WebSocket connection failed error message).

Am I doing something wrong here? 🤔

[nsarrazin]

@dylancl No problem, if you have an issue I want to help!

Just as a last check, can you try rebuilding with docker compose build --no-cache ? if the issue persists, I'll re-open it no problem.

[vnktsh]

I'm getting the same error too. Stuck on LOADING.

Running on Mac M2 Max 64gb RAM, 30B parameter.

This is the first run after downloading the repo, not likely cache issue.

[nawalnew]

Hello, had the same issue.

Should use first: docker compose build --no-cache

Thank you need to lead the models somehow again. My downloaded Models werent there.

The Best way is to delete your folder and docker file. And then make all steps Again, worked then.

git clone https://github.com/nsarrazin/serge.git && cd serge

cp .env.sample .env

docker compose up -d
docker compose exec api python3 /usr/src/app/utils/download.py tokenizer 7B

I just tried 7B model. 13B Didnt work first try