v10.0: Large Prompts Timing Out

[korgano]

OS: Windows 11 Pro
Lemonade version: 10.0

When using Lemonade as an LLM provider for another application (in this case, AnythingLLM), large prompts keep failing to generate output due to a Curl timeout:

2026-03-13 20:17:49.599 [Info] (Server) Model already loaded: user.Magidonia-v4.2.0
2026-03-13 20:17:49.599 [Info] (Server) POST /api/v1/chat/completions - Streaming
2026-03-13 20:17:49.663 [Info] (Process) srv  params_from_: Chat format: peg-native
2026-03-13 20:17:49.664 [Info] (Process) slot get_availabl: id  3 | task -1 | selected slot by LRU, t_last = -1
2026-03-13 20:17:49.664 [Info] (Process) slot launch_slot_: id  3 | task -1 | sampler chain: logits -> ?penalties -> ?dry -> ?top-n-sigma -> top-k -> ?typical -> top-p -> min-p -> ?xtc -> temp-ext -> dist 
2026-03-13 20:17:49.664 [Info] (Process) slot launch_slot_: id  3 | task 0 | processing task, is_child = 0
2026-03-13 20:17:49.664 [Info] (Process) slot update_slots: id  3 | task 0 | new prompt, n_ctx_slot = 32000, n_keep = 16, task.n_tokens = 15868
2026-03-13 20:17:49.664 [Info] (Process) slot update_slots: id  3 | task 0 | n_tokens = 0, memory_seq_rm [0, end)
2026-03-13 20:17:49.664 [Info] (Process) slot update_slots: id  3 | task 0 | prompt processing progress, n_tokens = 2048, batch.n_tokens = 2048, progress = 0.129065
2026-03-13 20:18:14.192 [Info] (Process) slot update_slots: id  3 | task 0 | n_tokens = 2048, memory_seq_rm [2048, end)
2026-03-13 20:18:14.192 [Info] (Process) slot update_slots: id  3 | task 0 | prompt processing progress, n_tokens = 4096, batch.n_tokens = 2048, progress = 0.258130
2026-03-13 20:18:47.901 [Info] (Process) slot update_slots: id  3 | task 0 | n_tokens = 4096, memory_seq_rm [4096, end)
2026-03-13 20:18:47.902 [Info] (Process) slot update_slots: id  3 | task 0 | prompt processing progress, n_tokens = 6144, batch.n_tokens = 2048, progress = 0.387194
2026-03-13 20:19:23.204 [Info] (Process) slot update_slots: id  3 | task 0 | n_tokens = 6144, memory_seq_rm [6144, end)
2026-03-13 20:19:23.204 [Info] (Process) slot update_slots: id  3 | task 0 | prompt processing progress, n_tokens = 8192, batch.n_tokens = 2048, progress = 0.516259
2026-03-13 20:20:00.208 [Info] (Process) slot update_slots: id  3 | task 0 | n_tokens = 8192, memory_seq_rm [8192, end)
2026-03-13 20:20:00.208 [Info] (Process) slot update_slots: id  3 | task 0 | prompt processing progress, n_tokens = 10240, batch.n_tokens = 2048, progress = 0.645324
2026-03-13 20:20:38.786 [Info] (Process) slot update_slots: id  3 | task 0 | n_tokens = 10240, memory_seq_rm [10240, end)
2026-03-13 20:20:38.786 [Info] (Process) slot update_slots: id  3 | task 0 | prompt processing progress, n_tokens = 12288, batch.n_tokens = 2048, progress = 0.774389
2026-03-13 20:21:18.840 [Info] (Process) slot update_slots: id  3 | task 0 | n_tokens = 12288, memory_seq_rm [12288, end)
2026-03-13 20:21:18.840 [Info] (Process) slot update_slots: id  3 | task 0 | prompt processing progress, n_tokens = 14336, batch.n_tokens = 2048, progress = 0.903453
2026-03-13 20:22:00.223 [Info] (Process) slot update_slots: id  3 | task 0 | n_tokens = 14336, memory_seq_rm [14336, end)
2026-03-13 20:22:00.226 [Info] (Process) slot init_sampler: id  3 | task 0 | init sampler, took 2.24 ms, tokens: text = 15868, total = 15868
2026-03-13 20:22:00.226 [Info] (Process) slot update_slots: id  3 | task 0 | prompt processing done, n_tokens = 15868, batch.n_tokens = 1532
2026-03-13 20:22:42.931 [Info] (Process) srv  log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200
2026-03-13 20:22:49.680 [Info] (Process) srv          stop: cancel task, id_task = 0
2026-03-13 20:22:49.913 [Info] (Process) slot      release: id  3 | task 0 | stop processing: n_tokens = 15898, truncated = 0
2026-03-13 20:22:49.913 [Info] (Process) srv  update_slots: all slots are idle

This was not an issue with Lemonade v9.1+, and seems to be a reversion of the behavior from #572.

Proposed Solution: Surface the timeout value in the CLI and GUI, allowing the user to configure it as necessary when issues with long prompts occur.

[sofiageo]

I've encountered this today on Linux, there is already a cli option and an environment variable for this

https://lemonade-server.ai/docs/server/lemonade-server-cli/#timeout-configuration

[winmutt]

I am also experiencing this:

Mar 16 11:46:08  lemonade-server[342286]: 2026-03-16 11:46:08.515 [Error] (HttpClient) CURL error: Failed writing received data to disk/application
Mar 16 11:46:08  lemonade-server[342286]: 2026-03-16 11:46:08.515 [Error] (WrappedServer) Streaming request failed: CURL error: Failed writing received data to disk/application

However setting the global timeout does not seem to impact this, I've tried both the config file and passing the argument in. The setting doesn't seem to be logged so I am not sure if it was working. To be honest I am not sure if this log indicates that the agent (opencode) or backend (llama-server) timed out

[worldowner]

I'm also facing this issue. My subagents in opencode get interrupted in the middle of work with
"CURL error: Timeout was reached"
I've already set --global-timeout 3600 in lemonade startup command and

    "lemonade": {
      "npm": "@ai-sdk/openai-compatible",
      "options": {
        "baseURL": "http://127.0.0.1:8000/api/v1",
        "timeout": false,
      },

in the provider section of opencode's configuration file.

[winmutt]

This is so bad I have reverted. I tried all of the timeout settings I could find without luck.

[james-martinez]

I've had similar issues and added some configuration options. There still are some static timeouts assigned.

Libcurl error needs to be investigated
Failed writing received data to disk/application"

@winmutt
@worldowner
@sofiageo

Are you able to get any approximate time of the actual timeout?
My fix was to go over 5 minutes. 300s.
Once we can get the time I can look over the static timeouts and see if they match.

[james-martinez]

In the mean time you can run llama-server outside of lemonade. Thats my workaround

[sofiageo]

Actually my issue went away when I configured the timeout to be 0 instead of the default. So maybe it's something different for other people.

[winmutt]

Actually my issue went away when I configured the timeout to be 0 instead of the default. So maybe it's something different for other people.

I tried -1 and found out it was an unsigned int. Then I tried 999999 and it still timed out.

Unsure of exact timing, maybe 5 min? I can try to measure tomorrow as well as try 0.

[winmutt]

I upgraded to 10.0 and set the timeout to 0. Still timing out.

Mar 20 10:24:44 lemonade-server[73257]: 2026-03-20 10:24:44.312 [Info] (Process) slot update_slots: id  2 | task 3 | new prompt, n_ctx_slot = 262144, n_keep = 16, task.n_tokens = 61322
Mar 20 10:30:04 lemonade-server[73257]: 2026-03-20 10:30:04.817 [Error] (HttpClient) CURL error: Failed writing received data to disk/application

So right at 5min give or take a few seconds.

[james-martinez]

Added this pr. My thinking was
Default 300s
Greater than 0 or Less than 300s: 300s
0: Infinite
Greater than 300s: use that

Any input would be appreciated

[winmutt]

I am no longer confident that my experience is just lemonade 10.0. I am using opencode and just timed out with 9.4.1 with one of the two error messages I was seeing in 10.0. I am going to fix the opencode timeouts in 9.4.1 and then give 10.0 a try again w a 0 timeout.

I am on strix halo and prompt processing is slow. With this most recent error I get a SSE read timed out error but I now believe that to be from OC because I see the prompt still loading in lemonade.

[james-martinez]

I was getting timeouts on 10.0.0 but after setting LEMONADE_GLOBAL_TIMEOUT=0. I can run long sessions.

Many clients have their own timeouts too. So be aware.

[arunkumar-reddy]

I'm having the same issue with large prompts timing out on strix halo ( slow prompt processing ) when running with opencode or pi.
Is it a timeout from the agent ( like opencode ) or rather some other timeout within the lemonade server?, it would help if the log was more descriptive.