Timeout for AI Assistant (30 sec.)

[HeikoMarkgraf]

Hi,
First of all, fantastic tool and it became in short time to my favorite Task tracker :-)
Now I want to push it to the next level.

I tried to connected my Mindwtr Desktop app to a local running language model. But as I don't have a high power computer the tokens/sec rate is quite low and the AI work takes some time. Also I recognized that larger models tent to create more output tokens (~1500) then smaller one (~200) for the "Break down task".
How ever, a 0.8B model with 60Token/Sec is able to answer the AI Assistant within the 30 Sec. timeout. All larger models (2B, 4B, 12B where the rate goes down to 30, 15, smaller 10 Token/Sec and the create tokens up to more then 1000) can't answer within the 30 Sec. timeout (on my machine). So it runs into the timeout and then tries 2 times again which takes 3 times 30 Sec. without a successful result.

From the source code it looks like the 30 sec. are the hardcoded default timeout but I could not spot a way to change this timeout to e.g. 60 or 120 seconds.
Is there a way to change this timeout?
If not, could it be added to the AI Assistant Setting so that it can be controlled specially for local / slow LLMs?

Thanks for your help

[dongdongbh]

Add a fix 09f157c.

There wasn't a timeout setting before. In the next release, desktop and mobile will have AI assistant → Advanced → Request timeout, with 30, 60, 120, or 300 seconds. Timed-out or cancelled requests won't restart automatically. Please try 120 seconds with your local model and let me know how it goes.

[HeikoMarkgraf]

Hi, amazing how fast you got this timeout-setting into the product :-) It works.
But the larger AI models still generates a large token number and still runs into timeouts just waits longer (with 120s with more then 4000 tokens). Which sounds a lot for a simple Task break down. But maybe I have to play around with the extra request parameter to send some "settings" to the AI model to give more "short / efficient" answers.
I have to say I'm quite new to local LLMs. So maybe the next steps is to work on some of the parameters via the extra request parameters. I will give it a try.

[HeikoMarkgraf]

Hi,
Here an update for all who are using llama.cpp local LLM with this tool. I tested it with a Qwen3.5-4B model and without any extra request parameter the answer of the LLM was just running to the max of the token context and mostly into the timeout (even with 120s). with the following extra request parameters, the LLM is answering in reasonable time (suggestion 6.45s and break down 37s) with 160 tokens answer:
{ "adaptive_decay": 0.9, "adaptive_target": 0, "add_bos_token": true, "auto_max_new_tokens": true, "ban_eos_token": false, "dry_allowed_length": 2, "dry_base": 1.75, "dry_multiplier": 0, "dry_sequence_breakers": "\"\\n\", \":\", \"\\\"\", \"*\"", "dynamic_temperature": false, "echo": false, "enable_thinking": false, "encoder_repetition_penalty": 1, "frequency_penalty": 0, "max_tokens": 512, "max_tokens_second": 0, "min_p": 0, "mirostat_eta": 0.1, "mirostat_mode": 0, "mirostat_tau": 5, "preserve_thinking": false, "repetition_penalty": 1, "repetition_penalty_range": 1024, "sampler_priority": "repetition_penalty\npresence_penalty\nfrequency_penalty\ndry\ntop_n_sigma\ntemperature\ndynamic_temperature\nquadratic_sampling\ntop_k\ntop_p\ntypical_p\nepsilon_cutoff\neta_cutoff\ntfs\ntop_a\nmin_p\nadaptive_p\nmirostat\nxtc\nencoder_repetition_penalty\nno_repeat_ngram", "seed": -1, "stream": false, "temperature": 0.1, "temperature_last": false, "top_k": 0, "top_p": 0.95, "truncation_length": 8192, "typical_p": 1, "xtc_probability": 0, "xtc_threshold": 0.1 }

[HeikoMarkgraf]

@dongdongbh I don't know where a good place would be (e.g. example in documentation) to make this available to others easily.

[dongdongbh]

I've added a local-model tuning section to the docs, with a link to your full Qwen3.5-4B configuration. It includes a smaller starting example and explains token limits and the differences between servers' thinking settings. Thanks for sharing your results!