DSH | dsh-fetch-timeouts | Stop slow local models (Ollama, LM Studio) from being cut off at 5 minutes

[d3vmeh]

Project URL:
https://github.com/d3vmeh/dsh-fetch-timeouts
npm: https://www.npmjs.com/package/dsh-fetch-timeouts

Introduction:

If you run DSH against Ollama or LM Studio and long turns die at exactly 5:00 with Failure reason: terminated (or UND_ERR_BODY_TIMEOUT / UND_ERR_HEADERS_TIMEOUT), even after raising streamIdleTimeoutMs, this is the missing piece. Node's built-in fetch gives up when a server sends no headers for 300 seconds, or no body bytes for 300 seconds, and DSH has no setting for those two timers. Ollama and LM Studio send nothing while a model thinks or generates a large tool call (for example a whole file for write), so the connection is cut before anything DSH controls gets a say. llama.cpp users are usually fine because llama-server pings every 30 seconds.

dsh-fetch-timeouts raises those two timeouts for the whole DSH process (default 30 minutes each). It replaces the node_modules patch people have been applying by hand in #3157 and #4518.

How it integrates with DSH:

A tiny cordis plugin (about 40 lines). At startup it installs an undici Agent with your timeouts as Node's global fetch dispatcher, and restores the previous one when unloaded. If NODE_USE_ENV_PROXY is set it installs undici's proxy-aware agent instead, so HTTP_PROXY, HTTPS_PROXY and NO_PROXY keep working. It ships a dsh.bundle patch, so installation is one line:

dsh plugin --profile web add dsh-fetch-timeouts

The defaults are enough for most people. To change them, in ~/.dsh/profiles/web/cordis.patch.yml:

- id: fetch-timeouts
  config:
    headersTimeoutMs: 3600000   # 0 disables the timer
    bodyTimeoutMs: 3600000

Also raise DSH's own watchdog on the provider route, or it fires first:

llm-pi-ai:
  providers:
    ollama:
      streamIdleTimeoutMs: 1800000
      timeoutMs: 1800000

What you should know:

• It is process-wide: every fetch in the DSH host gets the longer limits, so a genuinely dead connection takes longer to notice. Fine on a single-user machine.
• It is a stopgap. DSH's pi-ai dependency already accepts a custom fetch; when DSH exposes these timeouts itself, this plugin becomes unnecessary.
• Tested end to end against a stub server that stays silent for 5.5 minutes: without the plugin DSH gives up at 302 s; with it the turn completes. Node 22 with undici 8; not yet tested on Windows.

Related: dsh-llm-gate (#4995) queues requests so a single-slot llama.cpp is not overloaded, and dsh-context-budget (#5078) keeps the context at a size the GPU handles well. This one is the third piece for local models: the HTTP layer.

MIT License. Feedback welcome!

> Unofficial project, independently developed and maintained by community members.

[argszero]

Verified the tarball (dsh-fetch-timeouts@0.1.0) against the current harness source (0.1.2-alpha.1 = cd5ef81; 0a53fb5 is a version-only bump, same code). Your diagnosis holds, plus one structural detail and two cross-links:

Why it dies at exactly 5:00. Node's fetch (undici) Agent defaults are headersTimeout: 300_000 and bodyTimeout: 300_000 — confirmed. And the harness's own DEFAULT_STREAM_IDLE_TIMEOUT_MS is also 300_000 (packages/llm/llm-pi-ai/src/config.ts:43). The two timers coincide, which is why every report shows the same wall-clock cutoff. streamIdleTimeoutMs/timeoutMs do exist (config.ts:158-162, :332) but they're the pi-ai-layer stream watchdog, not the undici socket timers — raising only the harness side can't stop undici from cutting the socket first. Your framing ("DSH has no setting for those two timers") is correct.

Cross-link #5118 (global dispatcher). Your plugin's mechanism is a deliberate setGlobalDispatcher (process-wide, restored on unload with a no-clobber guard — clean). Two interactions worth knowing:

1. import('undici') itself has a global-dispatcher side effect in proxy environments: [Bug 修复] web_fetch 加载 undici 污染全局 dispatcher，导致代理环境下"模型不可用" #5118 documents that loading undici replaces Node's proxy-aware dispatcher for globalThis.fetch — the shared transport for the LLM adapter and web_search — until restart (NODE_USE_ENV_PROXY=1 + corporate proxy + one failed web_fetch ⇒ "model unavailable"). Your plugin also imports undici at module top, so it sits in that same module-load class; your immediate setGlobalDispatcher in apply() then overwrites whatever the import clobbered, so the plugin's own window is consistent — but the no-clobber restore guard (getGlobalDispatcher() === agent) can be defeated if that import side effect mutates the symbol between your apply() and unload.
2. web_fetch does not use the global dispatcher: requestPinned builds a per-request pinned Agent passed via the dispatcher option (packages/web/web-fetch-http/src/network.ts:180-186). So your plugin neither extends nor shortens web_fetch's timers, and it doesn't touch the SSRF pinned-lookup — and the reverse holds, a web_fetch failure cannot clobber your agent.

One tradeoff to state plainly: with 30-minute timers, a hung LLM provider stalls the whole host until either the pi-ai watchdog fires (defaults 300s, so normally it wins) or your timer expires. After raising streamIdleTimeoutMs as you recommend, the pi-ai watchdog becomes the only backstop — worth a doc sentence so users understand the failure-mode shift.

The end state is indeed the pi-ai custom-fetch route — nothing in the harness source passes a custom fetch into pi-ai today, so this plugin is currently the only lever. Good stopgap; the #5118 interplay is the one thing to watch.

[d3vmeh]

Thanks for the source-level check. Two of the three I have folded into the README: web_fetch is unaffected because it pins its own per-request Agent (web_search does use the global dispatcher, so it is covered), and the backstop shift once streamIdleTimeoutMs is raised is now stated plainly.

On the #5118 interplay: I looked at undici 8's global.js. It only installs its default Agent when no global dispatcher exists yet (if (getGlobalDispatcher() === undefined)), and setGlobalDispatcher also writes the legacy undici.globalDispatcher.1 symbol that older copies read. So a later import('undici') by another plugin cannot replace the plugin's agent after apply(), and the restore guard holds. What the import does change is the window before apply(): between module load and apply() the dispatcher is undici's plain Agent, which is the #5118 situation, but apply() runs immediately and with NODE_USE_ENV_PROXY set it installs EnvHttpProxyAgent, so proxy users end up better off than with the plugin absent. Anyone running a non-8 undici copy alongside would be in different territory; I have not tested that.

Agreed the real fix is a custom fetch passed into pi-ai; until then this is the lever.