bug: It's tough to get started with a 4k context on my Windows computer

[louis-jan]

Describe the bug
I get a very slow response and the entire OS becomes laggy when generating a response with a 4k context size (~30s TTFT). After changing the context size to 1k, it works quickly and smoothly (<1s TTFT). That was a very bad experience during the onboarding of a new user.

A better default medium context size would be ideal.

Expected behavior
The newly created thread should have the default context size set to 1024 or 2048 (depends on user's device specs).

Environment details

• Windows 11
• CPU: i5 10th
• GPU: RTX 2060 6GB VRAM
• RAM: 32GB

Logs
If the cause of the error is not clear, kindly provide your usage logs:

• tail -n 50 ~/jan/logs/app.log if you are using the UI
• tail -n 50 ~/jan/logs/server.log if you are using the local api server
  Making sure to redact any private information.

Additional context
Might need to add continue button support when the response reaches the limit.

[Van-QA]

tested for the case of default context length 2048 for the imported model ✅