cannot start new chat after the update

[MirProg]

Bug description

the start new chat button is unclickable update the updates made to the repo

Steps to reproduce

Start docker
go to http://localhost:8008/

Environment Information

docker v 4.17.1
Windows 11 Pro

Screenshots

Relevant log output

No response

Confirmations

• I'm running the latest version of the main branch.
• I checked existing issues to see if this has already been described.

[nsarrazin]

Looks like your models are not loaded. Did you download one ?

[skyface753]

same problem. Already downloaded a model with
docker compose exec serge python3 /usr/src/app/api/utils/download.py tokenizer 7B

[MirProg]

Looks like your models are not loaded. Did you download one?

yes, I do have the models downloaded, it was working before the update, but now it doesn't. Also, I am unable to run it on my GPU when after installing stuff to run it on GPU

[RahulVivekNair]

Yeah, I have the same problem

[mrg358]

Same problem. Fresh deploy, latest release. After click to Start New Chat serge throws a 405 Method Not Allowed in log output.
My instance is behind Nginx Proxy Manager. Here is the related log output.

"GET /chat?temp=0.1&top_k=50&max_length=256&top_p=0.95&ctx_length=512&repeat_last_n=64&model=ggml-alpaca-7B-q4_0.bin&n_threads=4&repeat_penalty=1.3&init_prompt=Below+is+an+instruction+that+describes+a+task.+Write+a+response+that+appropriately+completes+the+request.+The+response+must+be+accurate%2C+concise+and+evidence-based+whenever+possible.+A+complete+answer+is+always+ended+by+%5Bend+of+text%5D. HTTP/1.1" 405 Method Not Allowed

[FenarkSEC]

I'm encountering the exact same issue with Nginx as an IngressController in K8S. As soon as I try to put an ingress in front of the serge service (reverse proxy to http://serge:8008) with a TLS certificate I get a 405 Method not allowed.

[include]

+1

[voronind-com]

have same issue when using nginx proxy with TLS. direct 8008 access works.
maybe it is related to address name? it expects localhost, but gets whatever proxy is passing.

also I get proxy errors when my service in docker-compose is not called 'serge'.

[nsarrazin]

Hey everyone! I might have improved the situation with the latest version on main. Can you pull, rebuild and see if it fixed things ?

[dontflipmybits]

Thanks main works(for me). It's actually the first container build that has worked for me thus far. Worked flawlessly.

[voronind-com]

@nsarrazin yup, all works fine now. thank you very much! 💛

[FenarkSEC]

I've just tried with the new released production image, it now works on K8S with a reverse proxy and TLS like a charm !
@nsarrazin I think the issue can be closed :)

[nsarrazin]

Good to hear, feel free to open it if you have issues again

[include]

<3 working :) #FunResumed