You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug
I get a very slow response and the entire OS becomes laggy when generating a response with a 4k context size (~30s TTFT). After changing the context size to 1k, it works quickly and smoothly (<1s TTFT). That was a very bad experience during the onboarding of a new user.
A better default medium context size would be ideal.
Expected behavior
The newly created thread should have the default context size set to 1024 or 2048 (depends on user's device specs).
Environment details
Windows 11
CPU: i5 10th
GPU: RTX 2060 6GB VRAM
RAM: 32GB
Logs
If the cause of the error is not clear, kindly provide your usage logs:
tail -n 50 ~/jan/logs/app.log if you are using the UI
tail -n 50 ~/jan/logs/server.log if you are using the local api server
Making sure to redact any private information.
Additional context
Might need to add continue button support when the response reaches the limit.
The text was updated successfully, but these errors were encountered:
Describe the bug
I get a very slow response and the entire OS becomes laggy when generating a response with a 4k context size (~30s TTFT). After changing the context size to 1k, it works quickly and smoothly (<1s TTFT). That was a very bad experience during the onboarding of a new user.
A better default medium context size would be ideal.
Expected behavior
The newly created thread should have the default context size set to 1024 or 2048 (depends on user's device specs).
Environment details
Logs
If the cause of the error is not clear, kindly provide your usage logs:
tail -n 50 ~/jan/logs/app.log
if you are using the UItail -n 50 ~/jan/logs/server.log
if you are using the local api serverMaking sure to redact any private information.
Additional context
Might need to add continue button support when the response reaches the limit.
The text was updated successfully, but these errors were encountered: