Describe the feature
Hi, we've been digging into the experimental durable event iterator package recently, and I just noticed you're actively working on it in https://github.com/unnoq/orpc/pull/965, so I thought to maybe see what you think of the thoughts below.
We're really liking the relatively simple setup w type safety, from client <-> durable object, for both events and rpc calls.
However, we don't need the "durable" (sqlite) functionality - at least not how it's currently implemented. What do you think about maybe making all of the sqlite stuff opt-in, and exposing an onConnect method?
We've got the classic stream llm tokens use case. Stream tokens over the socket while it's in flight, and once the stream is done we squash the parts and and persist them to postgres. So while the llm call is ongoing, the stream is the source of truth for the client, and once the llm call is done postgres becomes the source of truth for that message.
So on the durable object side we just need some simple in memory state for messages that are actively streaming, which we could manage if we had an onConnect method and a means of setting our own event ids etc, like the base event iterator stuff.
Very rough example:
class MyDurableObject {
#inMemoryEvents: Events[] = [];
onConnect(ws: WebSocket, { context, lastEventId }) {
const relevantEvents = getThem(lastEventId); // our logic for getting the events
for (const evt of relevantEvents) {
this.dei.websocketManager.publishEvent([ws], evt);
}
}
// our publish method
#publish(evt: Event) {
if (evt.type === 'llm-end') {
// cleanup relevant #inMemoryEvents for this message.. it's done and will be served from postgres now
} else {
this.#inMemoryEvents.push(evt);
}
// set our own event ids, like base sse approach
this.dei.websocketManager.publishEvent(this.ctx.getWebSockets(), withEventMeta(evt, { id: evt.id }));
}
}Or maybe it would be enough if we could pass our own DurableEventIteratorObjectEventStorage in, I'm not sure.
Overall the durable-event-iterator package brings a lot more to the table DX wise than just the "durable"/sqlite persistence piece, and we'd love to be able to use all of that without the sqlite insert/delete delete operations on every event, which is overkill for our use cases.
Side note: the RpcTarget approach is super nice because it avoids having to subclass - thus won't conflict if one is already extending something else, etc. And can still support migrations, etc - example here -> https://github.com/benallfree/dofs/blob/main/packages/dofs/src/Fs.ts.
Additional information
Describe the feature
Hi, we've been digging into the experimental durable event iterator package recently, and I just noticed you're actively working on it in https://github.com/unnoq/orpc/pull/965, so I thought to maybe see what you think of the thoughts below.
We're really liking the relatively simple setup w type safety, from client <-> durable object, for both events and rpc calls.
However, we don't need the "durable" (sqlite) functionality - at least not how it's currently implemented. What do you think about maybe making all of the sqlite stuff opt-in, and exposing an
onConnectmethod?We've got the classic stream llm tokens use case. Stream tokens over the socket while it's in flight, and once the stream is done we squash the parts and and persist them to postgres. So while the llm call is ongoing, the stream is the source of truth for the client, and once the llm call is done postgres becomes the source of truth for that message.
So on the durable object side we just need some simple in memory state for messages that are actively streaming, which we could manage if we had an onConnect method and a means of setting our own event ids etc, like the base event iterator stuff.
Very rough example:
Or maybe it would be enough if we could pass our own
DurableEventIteratorObjectEventStoragein, I'm not sure.Overall the
durable-event-iteratorpackage brings a lot more to the table DX wise than just the "durable"/sqlite persistence piece, and we'd love to be able to use all of that without the sqlite insert/delete delete operations on every event, which is overkill for our use cases.Side note: the RpcTarget approach is super nice because it avoids having to subclass - thus won't conflict if one is already extending something else, etc. And can still support migrations, etc - example here -> https://github.com/benallfree/dofs/blob/main/packages/dofs/src/Fs.ts.
Additional information