Paginated conversation snapshots for long-running agent chats

[fabian-hiller]

RFC

Summary

Allow history() to return a bounded slice of a conversation, so a client can load the newest messages first and fetch older ones on demand.

Background & Motivation

history() always returns the whole materialized conversation. FlueConversationHistoryOptions is { signal } and AgentConversationObserveOptions is { live, signal, backoffOptions }, so neither read can be bounded. observe() hydrates by calling history() internally, so a transcript UI receives every message the conversation has ever held on every page load.

For an agent conversation that stays open for days, this grows without limit. One of our conversations is 68 messages and a 175.5 KB snapshot, of which 81.4 KB (46%) are dynamic-tool parts. The newest 10 messages alone are 78.3 KB. 68 messages is short for us. Ours run into the hundreds, and both the payload and the cost of rendering it grow linearly, since every message is a markdown parse plus syntax highlighting if it contains code or a table.

We already render only the newest messages and load older ones when the user scrolls up. This fixed the rendering cost, but not the transport cost, because the whole snapshot has arrived before the client decides what to render.

This was requested in #339 and closed as not planned, with the note that backward pagination would be a new paginated snapshot feature and not a fix to the old durable event stream API. This is that feature request, against the conversation API.

Goals

• Read the newest N messages without transferring the whole conversation
• Load older messages on demand when the user scrolls up
• Keep live resumption working, since offset and incarnation are stream checkpoints and do not depend on how many messages a snapshot carries
• Stay additive so that an unbounded history() keeps its current behavior

Example

This is a first idea to make the request concrete, not a proposal:

const page = await client.history({ limit: 50 });
const older = await client.history({ before: page.cursor, limit: 50 });

observe() would also need a way to hydrate from a bounded read.

There is more to it than slicing a list. A client that only receives the newest messages loses everything it derives from the full transcript, and I am not sure yet what the right design is. If you are interested in this, I would like to take a closer look and come back with a more well-thought-out plan.

[matthewp]

Implemented in #767