Fix long-running SSE client disconnect cleanup

[davide221]

Fixes long silent-prefill requests across Pi and the other real-client harnesses. The server now emits SSE heartbeat comments, detects peer closure independently of writes, and propagates cancellation to backend prefill/decode boundaries. Pi HTTP idle timeout is disabled in its generated settings, and all real-client launcher deadlines now default to a configurable hour. Verified on Lucebox6: 329/329 server unit tests, both harness regressions, a real DS4 heartbeat request, and an intentional disconnect that stopped at the next prefill boundary with zero output tokens.

[cubic-dev-ai]

1 issue found across 27 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="server/src/common/model_backend.h">

<violation number="1" location="server/src/common/model_backend.h:86">
P1: Client disconnect cancellation still does not stop long-running prefill for several server backends, so the new probe does not provide the advertised cleanup behavior across the supported model paths. `DeepSeek4Backend::do_prefill` polls `io.is_cancelled()`, but `Qwen3Backend::do_prefill` performs its chunk loop without polling, `Gemma4Backend::do_prefill` explicitly discards `io`, and `Qwen35Backend::do_prefill` also explicitly discards it; the Qwen35-MoE hybrid and Laguna manual prefill loops likewise have no probe boundary. Adding the probe to each chunk/layer boundary (including the layer-split prefill loop) would make disconnects terminate prefill consistently.</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

P1: Client disconnect cancellation still does not stop long-running prefill for several server backends, so the new probe does not provide the advertised cleanup behavior across the supported model paths. DeepSeek4Backend::do_prefill polls io.is_cancelled(), but Qwen3Backend::do_prefill performs its chunk loop without polling, Gemma4Backend::do_prefill explicitly discards io, and Qwen35Backend::do_prefill also explicitly discards it; the Qwen35-MoE hybrid and Laguna manual prefill loops likewise have no probe boundary. Adding the probe to each chunk/layer boundary (including the layer-split prefill loop) would make disconnects terminate prefill consistently.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At server/src/common/model_backend.h, line 86:

<comment>Client disconnect cancellation still does not stop long-running prefill for several server backends, so the new probe does not provide the advertised cleanup behavior across the supported model paths. `DeepSeek4Backend::do_prefill` polls `io.is_cancelled()`, but `Qwen3Backend::do_prefill` performs its chunk loop without polling, `Gemma4Backend::do_prefill` explicitly discards `io`, and `Qwen35Backend::do_prefill` also explicitly discards it; the Qwen35-MoE hybrid and Laguna manual prefill loops likewise have no probe boundary. Adding the probe to each chunk/layer boundary (including the layer-split prefill loop) would make disconnects terminate prefill consistently.</comment>

<file context>
@@ -80,6 +80,11 @@ constexpr bool park_target_includes_draft_model(ParkTarget target) {
+// Return true when an in-flight request should stop. Backends poll this at
+// their existing prefill/decode cancellation boundaries so cancellation does
+// not depend on filling the socket's send buffer first.
+using CancellationProbe = std::function<bool()>;
+
 // Inference observer callback for live status updates. Called by backends
</file context>

[cubic-dev-ai]

All reported issues were addressed across 12 files (changes from recent commits).

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>