Repository navigation
Replies: 1 comment
|
Thanks for writing this up so clearly, and for turning #879 around so quickly. We measured before deciding. We took the 10 longest live sessions on our own website agent (92 visitor turns, 97 agent replies), rebuilt every frame the single-event stream sends from the stored events, and compared that with deltas. The numbers are an upper bound: one frame per chunk, uncompressed.
So deltas would save about 23% of what a visitor downloads, roughly 1 KB per reply. The saving doesn't come from quadratic growth: the median reply arrives in a single chunk. Almost all of the overhead is the The only streams that do grow quadratically are reasoning streams (about 40 chunks each). Guests and customers never receive them, because redaction strips the chunks and the widget doesn't follow a status event without them. That cost only applies to admin callers following reasoning live. Our take: option 1 for now, no delta format. We're inclined to leave it as it is. A frame that carries the whole message so far makes UI work simpler in many cases: a client can just render the latest frame, without having to join chunks or work out which update it is applying. A second frame shape is a permanent cost for every client and every redaction path, and the measured saving doesn't justify it. Two narrower fixes would recover most of it without a new format, and we'd welcome PRs for either:
On your questions:
On option 3: in the merged code, resuming at a chunk cursor skips frames the client has already seen, but the next frame is still the whole event so far, so it saves frames rather than bytes. It still needs no new format, so we're open to it if mid-message reconnects turn out to be common. Leaving this open for anyone with data to add. |
Uh oh!
There was an error while loading. Please reload this page.
PR #879 adds resumable event streams: an
id:on every frame,Last-Event-IDresumption, and keep-alives. It also added a second wire format for streaming a single event, and @kichanyurd asked to settle that on its own. This is the place.The problem. A streaming event's
data.chunksgrows while the response is generated, and every frame repeats every chunk so far. A long answer makes the stream quadratic in bytes, which matters most for the clients that stream longest.What the first attempt did, and why it was withdrawn.
?sse=true&delta=truesent only the chunks added since the previous frame, after one full frame, withid: <offset>:<chunk_count>as the cursor. Two problems: it is a second frame shape that every client and every redaction path has to know about, and it readchunksoff the stored event, so a caller withoutREAD_EVENT_REASONINGcould get reasoning text by asking for deltas.Options I can see.
Last-Event-ID: <offset>:<count>), so a reconnect does not replay the prefix. Helps reconnects, not the steady state.Questions that decide it.
wait_for_completion=trueenough for the long cases?idkeep naming the chunk count (so a resume is unambiguous), or does the frame carry an explicit "last" flag?I am happy to implement whichever direction this lands on; the resumability part does not depend on it.
All reactions