Http Query subscriptions API

[gedw99]

The Http Query API looks really nice. LiteFS is really getting there at making a complex race condition area blindingly simple.

i wonder if subscriptions can be supported over the http transport using SSE or WS. Only saying this because the underlying litefs system knows what has changed and so subscriptions are then possible .

Don’t want to turn it into a graphql monster . But it could just be able to track presence ( I.e that you made a query a minute ago ) and then know that user x make a query and is still connected and so here is the changes to those records that you queries a minute ago. It’s essentially a simple CDC protocol. Again NOT graphql. Someone could build graphql on top of that’s what they wanted.

The presence aspect could also just be a TTL. You make a query with a TTL x-header and so the caller is in control of how long each query acts like a subscription.

The http proxy can then just keep a lookup table that maps users to queries and their TTL.

When a record changes it then just works out who is affected users via the lookups and then calculates the difference.

How to calculate the difference though :) various approaches exist . There is probably sone underlying litefs primitives that could be employed perhaps.

Ah I was doing so well until I hit that last bit :)

[benbjohnson]

LiteFS is a physical replication tool so it only works in pages. It's probably possible to work out the row change set by parsing the pages and using PTR pages to work out which parent pages & tables those pages belong to but it's a fairly complicated problem. Additionally, LiteFS would need to parse out the SQL and materialize updated rows in the same format.

While it's not as fancy, I think that attaching event data (#20) allows applications to implement something similar. It allows applications to attach events to write transactions and then replicas get notified when they apply those transactions and the application instance on the replica can requery.