Skip to content

fix(web): recover from quiet or stalled connections - #370

Open
testikun wants to merge 2 commits into
openpi-dev:mainfrom
testikun:codex/issue-363-web-resilience
Open

fix(web): recover from quiet or stalled connections#370
testikun wants to merge 2 commits into
openpi-dev:mainfrom
testikun:codex/issue-363-web-resilience

Conversation

@testikun

@testikun testikun commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

Problem

The OpenPI Web event stream could remain silently half-open, and quiet streams gave the browser no bounded way to verify canonical state. API and prompt-admission waits were also unbounded, leaving users without actionable feedback when a connection stalled. This completes the remaining liveness/recovery slice of #363 and is intentionally separate from the state-regression work in #364.

Value

Web clients now detect silent failures, periodically reconcile quiet connections with the canonical snapshot, and retain prompt drafts with explicit localized feedback when admission times out.

Approach

  • Send bounded SSE comment heartbeats per connected client without advancing the event cursor, and clean up each timer on disconnect or Host shutdown.
  • Use heartbeat cadence for low-frequency, single-flight snapshot reconciliation and a read watchdog for stalled streams.
  • Bound ordinary API and prompt-admission requests with AbortController; preserve failed prompt drafts and surface distinct pending, timeout, error, and reconnecting feedback.
  • Keep all recovery behavior within the existing Host/event/snapshot lifecycle rather than adding another connection-state store.

Validation

Impact

  • User-visible behavior: stalled requests and interrupted live updates now produce explicit English/Chinese feedback; failed prompt admission restores the draft.
  • Model-visible context/tools: none.
  • Runtime/lifecycle: one unref'd heartbeat timer per connected SSE client, bounded by the existing eight-client cap and cleared during disconnect/shutdown.
  • Persisted config/data: none.
  • Compatibility/risk: protocol-compatible SSE comments; snapshot cursors are unchanged by heartbeats. Reconciliation adds one snapshot request per four quiet heartbeats.

@tt-a1i tt-a1i left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

缝是对的,请先变绿;用这份替代 #377/#364

只给 /api/prompt 准入加超时,SSE comment 不推进游标,心跳在 disconnect/shutdown 时清掉 — 这是对的全局 30s api()#377)不该做的事。#364 改的同一批 running 行会被这份盖掉,请把那份并进来。

应修:CI 在 bun run test 中途取消。readEventChunk(45s)加重连,在只心跳、永不 settle 的 mock 上可能挂住套件。请先有一次绿的测试再合。

@tt-a1i tt-a1i left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact-head review: 72b334d

[P1] Timing out prompt admission creates an uncertain mutation that the UI presents as a safe retry. web/ui/app.js:803-825 aborts POST /api/prompt after 30 seconds, removes the optimistic row, restores the draft, and reports failure. The server path at web/host/web-host.ts:492-527 has already generated a new command id and may still complete runtime.sendPrompt(); it neither binds client disconnect to cancellation nor accepts a client idempotency key. Retrying the restored draft therefore creates another command id and can execute the same model/tool turn twice.

Please make admission retry-safe: use a stable client-generated idempotency key with server-side deduplication and a queryable receipt, or expose timeout as unknown/pending and reconcile canonical state before allowing retry. Add a test where the client times out just before the first admission completes, retries, and proves only one turn is admitted.

Verification boundary: exact-head cross-layer Standards + Spec review. Existing liveness tests cover bounded waiting, not the timeout/late-success/retry race.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants