Python: [Bug]: Python: the AG-UI FastAPI host has no SSE keepalive, so long output-silent runs are killed by the client's idle timeout

[antsok]

Description

1. Summary

add_agent_framework_fastapi_endpoint streams the run as SSE, but the response generator yields only real AG-UI events — it never interleaves a keepalive comment during idle gaps between events. When a turn goes output-silent for longer than a downstream idle timeout — most commonly a reasoning model "thinking" on a large context, and/or an app that strips reasoning deltas from the stream — the SSE connection carries zero bytes for that whole window. Any idle-timeout layer between the host and the browser (a reverse proxy, a load balancer, or a Node/undici fetch in a proxying runtime) then closes the still-healthy stream, and the run dies with no assistant message.

This is a robustness gap, not a bug in the run itself: the agent is alive and progressing the entire time; only the transport is silent. The standard remedy — a periodic SSE comment (:-prefixed line, ignored by every compliant SSE parser) — is not implemented and not configurable.

2. Symptom — observed live

Full-stack run (harness agent + Microsoft Learn MCP over AG-UI, consumed by a CopilotKit/Next.js frontend whose server-side runtime proxies to this host with Node fetch/undici). Provider ollama, model minimax-m3:cloud (a reasoning model). An Execute-mode "write the exhaustive deliverable" prompt ran its research rounds fine, then entered a long reasoning round on a ~107k-token context. Because the model thinks server-side before emitting output — and the app strips reasoning from the web stream — no SSE bytes flowed for minutes. At ~391s the browser reported:

[CopilotKit] Agent error: terminated   runtimeErrorCode: INCOMPLETE_STREAM

Host-side there was no RUN_ERROR, no exception, no backend error — the run was still waiting on the model. The ~300s of silence → terminate window matches undici's default bodyTimeout/headersTimeout (300000 ms) exactly. Nothing was truncated; the idle connection was simply dropped, and the turn ended with no answer.

3. Root cause — no keepalive anywhere on the AG-UI SSE path

In agent_framework_ag_ui/_endpoint.py:

Element	Keepalive?	Evidence
add_agent_framework_fastapi_endpoint(...) signature (≈:73)	No	11 parameters (app, agent, path, state_schema, predict_state_config, allow_origins, default_state, tags, dependencies, snapshot_store, snapshot_scope_resolver) — none for ping/keepalive/heartbeat/interval
event_generator() (≈:163)	No	async for event in protocol_runner.run(...): yield encoder.encode(event) — yields only real events; nothing is emitted during an idle gap
Response object (≈:211)	No	plain fastapi.responses.StreamingResponse(media_type="text/event-stream"); the Connection: keep-alive header it sets is HTTP-level, not an application SSE heartbeat, and X-Accel-Buffering: no only disables proxy buffering

So during a silent model round the generator is simply parked on the model call and emits no bytes — there is no timer-driven heartbeat to keep the stream warm, and no knob to enable one.

The fix is already in the dependency tree

agent-framework-ag-ui transitively installs sse-starlette 3.4.5, whose EventSourceResponse has a built-in ping parameter — it emits a : ping comment every N seconds (default 15s), exactly the mechanism needed (sse_starlette/sse.py EventSourceResponse.__init__(..., ping: Optional[int] = None, ...), ping: Ping interval in seconds (0 to disable). Default: 15.). The host uses the plain Starlette StreamingResponse instead, so this capability is present but unused.

4. Proposed feature

Ordered by preference; all are additive and change nothing about the AG-UI event contract (comments are invisible to compliant parsers).

Option A — emit EventSourceResponse with a configurable ping

Return sse_starlette.EventSourceResponse(event_generator(), ping=keepalive_seconds, ...) instead of StreamingResponse, and surface keepalive_seconds (e.g. default 15, 0/None to disable) on add_agent_framework_fastapi_endpoint:

add_agent_framework_fastapi_endpoint(app, agent, keepalive_seconds=15)

Smallest change, reuses an already-shipped dependency, and gets correct comment framing and disconnect handling for free.

Option B — interleave a keepalive comment in event_generator directly

Keep StreamingResponse but race the event stream against a timer and yield ": keepalive\n\n" after keepalive_seconds of no real event:

# pseudocode
while True:
    event = await next_event_or_timeout(protocol_runner, timeout=keepalive_seconds)
    yield encoder.encode(event) if event is not _IDLE else ": keepalive\n\n"

No new dependency, but re-implements what EventSourceResponse already does.

Option C — document the gap + expose the response for wrapping (minimal)

If neither ships soon, document that long silent turns require a host- or proxy-level keepalive, and/or let callers supply the StreamingResponse/headers so an app can wrap it. This is a stopgap, not a fix.

5. Impact

• Healthy long turns die with no answer and no error. Reasoning models on large contexts (long time-to-first-token), agents that strip reasoning deltas, and slow tool/model rounds all produce multi-minute silent gaps that exceed common idle timeouts (undici 300s; many proxies/LBs 60s).
• Provider-agnostic and silent. There is no RUN_ERROR to catch and no backend fault to blame; it looks like a collector or network problem, so it's easy to misdiagnose (we first mistook it for output-token truncation).
• Every AG-UI consumer that proxies over a timing-out transport is exposed — notably the CopilotKit Node runtime (undici), which is the reference web integration.
• No layer below the host will fix it. The AG-UI protocol defines no keepalive, and the reference consumer (CopilotKit) emits none either (see §7), so if the host doesn't keep the stream warm, nothing does.

6. Workaround

Application-side ASGI middleware that injects an ignored SSE comment (: keepalive\n\n) every 15s during idle gaps only (any real body resets the timer), wrapping the AG-UI app.

7. Prior art — keepalive is unowned across the whole stack

The gap isn't specific to this host; it's that no layer in the AG-UI stack claims responsibility for keeping a silent stream alive, which is exactly why the host is the right place to fix it.

• The AG-UI protocol defines no keepalive. The event spec and the @ag-ui/core EventType enum enumerate only lifecycle / text / tool-call / state / activity / reasoning / raw / custom events — there is no PING/HEARTBEAT/KEEPALIVE event, and the spec says nothing about SSE idle timeouts or long silent runs. Keepalive is deliberately a transport concern, so an SSE comment (invisible to the event vocabulary — @ag-ui/client's parser skips comment-only frames) is the only spec-compatible mechanism. Options A/B above are protocol-clean for exactly this reason.
• The reference consumer (CopilotKit) emits no keepalive either. Its Node runtime and @ag-ui/client HttpAgent drive the fetch with an AbortController fired only on explicit abort/unmount — no bodyTimeout override, no heartbeat to host or browser — so they inherit undici's 300s default. This is a known, still-open problem there:
  • CopilotKit #2059 — Timeout during long running tasks: an agent silent for 2+ minutes times out; the only mitigation offered is to call copilotkitEmitState() periodically — i.e. push a heartbeat burden onto every agent author. A host keepalive removes that burden agent-agnostically.
  • CopilotKit #3323 — Reasoning/Thinking events stall: silent reasoning phases stalling the stream.
  • CopilotKit #3208 — INCOMPLETE_STREAM: same client-side symptom from a different cause (a run that never emits its terminal event) — noted here so the two are not conflated.

Code Sample

Language/SDK

Python