bug: LLM stops responding after 3-4 messages

[0xSage]

Env: 4090s on linux x86_64

Convo history:

username
01:47 pm
tell me a lohng story

Llama-2-7B-Chat
01:47 pm
 Absolutely! I'd be happy to listen to a long story. Please go ahead and tell me anything that comes to mind. Is there a particular theme or topic you'd like to explore, or would you prefer me to suggest some prompts?

username
01:48 pm
no you tell me a long story about mermaids please

Llama-2-7B-Chat
01:48 pm
###  Sure! Here is a long story about mermaids:

username
01:48 pm
where did the story go?

Llama-2-7B-Chat
01:48 pm
[never responds]

Logs:
none - or rather the current logs are so busy that I cannot see what even went wrong

[louis-jan]

Not yet able to reproduce, will try more this wk. If you encounter the issue again, please go to jan-worker-1 service on your Docker and duplicate the log here.

[hiro-v]

I have this error too

Mac m1 pro
Git commit id: 4131a3b

2023-09-08 17:03:07 workerd/jsg/jsg.c++:136: error: took recursive isolate lock; kj::getStackTrace() = /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@1651d0f /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@1909503 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@192f32f /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@192f29b /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@2fc5d93 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@1e4d4b3 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@2f9efc3 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@2fe5fc3 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@2f9f27f /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@2fa04ab /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@194015f /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@2fa03a3 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@2f9c677 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@206d66f /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@2f9c97f /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@206d2fb /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@18b24bb /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@2f9c677 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@206cf7f /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@206cdf7 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@206d0bf /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@19babfb /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@1cdb17f /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@1c3ff6f /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@18d6f63 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@1c2f8ef /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@1c18ee7 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@1c190df /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@1c47c1f

-> After restarting, it seems working fine but on UI there is loading indicator waiting till infinity

[0xSage]

This is related to issue #120

Note: This is happening with all containers healthy

See: Private Zenhub Image

Logs:

++-- request sent
++-- request sent
++-- request sent
++-- request sent
++-- request sent
++-- request sent
++-- request sent
++-- request sent
[mf:inf] POST / 200 success (63681ms)
[mf:inf] POST / 200 success (7ms)
++-- request sent
++-- request sent
++-- request sent
[mf:inf] POST / 200 success (12194ms)
2023-09-10 10:26:48 
2023-09-10 10:25:17 workerd/server/server.c++:2518: error: Uncaught exception: kj/async-io-unix.c++:186: disconnected: remote.worker_do_not_log; Request failed due to internal error
2023-09-10 10:25:17 stack: /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@2fd61f7 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@2081024 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@2fb6de8 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@1aa33f0 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@1c3ee2c /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@1a3e844 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@1a3f4a7 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@18ffbd0 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@1900134 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@1900fa4 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@1901528 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@193fc54 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@18eaa8c /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@206d457 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@206c980 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@16328d0 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@1a3ea88 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@1a3ee1b /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@1901528 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@2033bf0 /usr/local/lib/node_modules/wrangler/node_modules/@cloudflare/workerd-linux-arm64/bin/workerd@2037068
2023-09-10 10:25:17 ✘ [ERROR] Uncaught (in response) Error: Network connection lost.
2023-09-10 10:25:17 
2023-09-10 10:25:17 

[dan-homebrew]

Follow-up: #143

[louis-jan]

No longer relevant, #143 fixed this