Skip to content

refactor(server): consolidate the SSE keepalive interval and its guard - #1133

Merged
inureyes merged 3 commits into
mainfrom
refactor/issue-1105-sse-keepalive-constant
Aug 13, 2026
Merged

refactor(server): consolidate the SSE keepalive interval and its guard#1133
inureyes merged 3 commits into
mainfrom
refactor/issue-1105-sse-keepalive-constant

Conversation

@inureyes

@inureyes inureyes commented Aug 13, 2026

Copy link
Copy Markdown
Member

Summary

The 15-second SSE keepalive interval was declared three times, once per surface, and the compile-time assertion that gives the number its meaning guarded only one of them. /v1/responses and the Anthropic-compatible surface could therefore be raised past a reverse-proxy idle timeout without the build failing. This consolidates the constant and moves the guard next to it, so it covers every consumer by construction.

What changed

  • src/server/streaming.rs: SSE_KEEPALIVE_INTERVAL_SECS is now documented as the single definition for every SSE surface, and the const _: () = assert!(SSE_KEEPALIVE_INTERVAL_SECS < 60, ...) moved here from streaming_tests.rs to sit directly under it. The module docs no longer restate 15; they point at the constant and record that the invariant now covers all surfaces.
  • src/server/streaming_responses.rs, src/server/streaming_anthropic.rs: local const KEEPALIVE_INTERVAL_SECS: u64 = 15; deleted; both import the shared constant. The two newtypes are untouched and stay distinct, which is the property refactor(server): consolidate the three duplicated SSE keepalive constants under one proxy-timeout assertion #1105 asks to keep: a route cannot attach another surface's keepalive, because the type it receives comes from the channel constructor it called. Only the constant and the assertion were duplicated.
  • src/server/streaming_tests.rs: the assertion and its now-unused import are removed, replaced by a comment recording where the invariant lives and why it moved.
  • src/server/router_front.rs: both KeepAlive::default() sites now build through SseKeepAlive::default_for_long_prefill(). Behaviourally identical today (KeepAlive::new() is 15s under axum 0.7.9), but it restores the design the newtype exists for: if the shared constant is ever lowered for proxy compatibility, these two streams would otherwise silently keep 15s.

Behaviour is unchanged. Every value was already 15 and still is.

Acceptance criteria

Criterion Evidence
Exactly one 15 literal for the SSE keepalive interval in src/server/ grep -rn "KEEPALIVE_INTERVAL_SECS.*=" src/server/ returns one line, streaming.rs
Changing it to 61 fails the build, attributable to the shared assertion Verified locally, then reverted. See below
No KeepAlive::default() call sites remain in src/server/ grep -rn "KeepAlive::default()" src/ returns four lines, all comment text in the two explanatory blocks left at the former call sites, and zero code
server::streaming tests green 25 passed, 0 failed

The 61 check, run against cargo check --profile test-fast --features cuda --lib and then reverted:

error[E0080]: evaluation panicked: SSE keepalive interval must be less than the 60s default used by most reverse proxies
  --> src/server/streaming.rs:77:15
   |
77 |   const _: () = assert!(
   |  _______________^
78 | |     SSE_KEEPALIVE_INTERVAL_SECS < 60,
79 | |     "SSE keepalive interval must be less than the 60s default used by most reverse proxies"
80 | | );
   | |_^ evaluation of `server::streaming::_` failed here

Before this change the same edit compiled cleanly for the Responses and Anthropic surfaces, because their copies were never named by any assertion.

Test plan

  • cargo test --profile test-fast --features cuda --lib server::streaming: 25 passed, 0 failed
  • cargo clippy --profile test-fast --features cuda --lib --tests -- -D warnings
  • cargo fmt --all -- --check
  • The 61 build-failure check above, reverted afterwards

The acceptance criterion names --features metal,accelerate. This is a Linux CUDA box and cannot build that feature set; the feature selection does not reach any of this code, which is backend-agnostic SSE plumbing. No CHANGELOG.md entry: nothing observable changes for a user.

Closes #1105


Review round

A review pass confirmed the refactor is behaviour-neutral against the vendored axum 0.7.9 source: KeepAlive::default() is KeepAlive::new(), which sets max_interval: Duration::from_secs(15) and the :\n\n comment event, so SseKeepAlive::default_for_long_prefill() overwrites max_interval with the same value and leaves the event alone. The two router_front sites emit the same frame on the same schedule.

It also found the guard's reach widened on a second axis that was not obvious going in. streaming_tests.rs is included under #[cfg(test)], so the assertion was const-evaluated only in test builds. As a module-level const _ in streaming.rs it is evaluated in every build, which is why the 61 check above reproduces under a plain cargo check --lib. Three surfaces instead of one, and all profiles instead of test-only.

One finding was that this PR made the SseKeepAlive rustdoc stale in the act of fixing the constants: it said the type is "constructed by sse_channel" and listed three consumers, while router_front.rs now constructs it directly and is a fourth. Fixed, along with two pre-existing inaccuracies in the same file (the sse_channel doc pointed at keepalive.0, which is private outside the module, and the module-doc lead-in presented the keepalive as something only sse_channel hands out).

The 15-second SSE keepalive interval was declared three times, once per surface: `SSE_KEEPALIVE_INTERVAL_SECS` in `streaming.rs` and a file-private `KEEPALIVE_INTERVAL_SECS` in each of `streaming_responses.rs` and `streaming_anthropic.rs`. The compile-time assertion that gives the number its meaning, that the interval stays under the 60s reverse-proxy idle default, named only the first of the three, so `/v1/responses` and the Anthropic-compatible surface could be raised past a proxy timeout without the build noticing. The module docs at `streaming.rs` state the invariant as a property of SSE generally rather than of one route family, so the narrow coverage was drift, not intentional scoping.

`streaming_responses.rs` and `streaming_anthropic.rs` now import the constant and their local copies are gone, leaving exactly one `15` literal for this interval in `src/server/`. The `const _: () = assert!(...)` moves from `streaming_tests.rs` to sit directly under the definition, so it covers every consumer by construction and needs no per-surface duplication. Verified by setting the constant to 61 and building: `error[E0080]: evaluation panicked: SSE keepalive interval must be less than the 60s default used by most reverse proxies`, pointing at `src/server/streaming.rs`, then reverted.

The three newtypes stay separate on purpose. They are distinct types so a route cannot attach another surface's keepalive; only the constant and the assertion were duplicated.

`router_front.rs` built its two SSE responses with `KeepAlive::default()`, which is behaviourally identical today because `KeepAlive::new()` sets a 15s interval under axum 0.7.9, but it defeats the design the newtype exists for: if the shared constant is ever lowered for proxy compatibility, those two streams would silently keep 15s. Both now go through `SseKeepAlive::default_for_long_prefill()`, and no `KeepAlive::default()` call site remains in `src/server/`.

Behaviour is unchanged. Every value was already 15 and still is.

Validated on GB10: `cargo test --profile test-fast --features cuda --lib server::streaming` is 25 passed and 0 failed, `cargo clippy --profile test-fast --features cuda --lib --tests -- -D warnings` is clean, and `cargo fmt --all -- --check` is clean. The acceptance criterion names `--features metal,accelerate`, which this Linux CUDA box cannot build; the feature set does not reach this code.

Closes #1105
@inureyes inureyes added type:refactor Code restructuring without changing functionality priority:low Low priority status:review Under review labels Aug 13, 2026
Review caught that this PR made the `SseKeepAlive` rustdoc wrong in the act of fixing the constants. It said the type is "constructed by `sse_channel`" and listed three consumers, but `router_front.rs` now constructs it directly at two sites and is a fourth consumer, since its streams do not come from `sse_channel` at all. The `Used by:` list is a convention this repo maintains across 20-odd rustdoc sites and deliberately tracks for staleness (see the report for PR #1120), so leaving it short would have been the exact defect that report was written about. The sibling list on `sse_channel` is untouched and still correct: `router_front` does not call it.

Two pre-existing inaccuracies in the same file, folded in while the lines were open. The `sse_channel` doc told handlers to attach via `keep_alive(keepalive.0)`, but the field is private outside the module and all seven attach sites use `into_inner()`, which the struct doc five lines above already said correctly. The module-doc lead-in presented the keepalive as something only `sse_channel` hands out, which stopped being true for two of the seven sites.

The report gains a paragraph on a second way the guard's reach widened, which was not obvious going in: `streaming_tests.rs` is included under `#[cfg(test)]`, so the assertion was const-evaluated only in test builds. As a module-level `const _` in `streaming.rs` it is evaluated in every build, which is why the 61 experiment reproduces under a plain `cargo check --lib`. Three surfaces instead of one, and all profiles instead of test-only.

No code changed. Validated on GB10: `cargo test --profile test-fast --features cuda --lib server::streaming` is 25 passed and 0 failed, clippy with `-D warnings` is clean, and `cargo fmt --all -- --check` is clean.

Refs #1105
@inureyes inureyes added status:done Completed and removed status:review Under review labels Aug 13, 2026
@inureyes
inureyes merged commit e685122 into main Aug 13, 2026
8 checks passed
@inureyes
inureyes deleted the refactor/issue-1105-sse-keepalive-constant branch August 13, 2026 18:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

priority:low Low priority status:done Completed type:refactor Code restructuring without changing functionality

Projects

None yet

Development

Successfully merging this pull request may close these issues.

refactor(server): consolidate the three duplicated SSE keepalive constants under one proxy-timeout assertion

1 participant