Feature Request: Event-based subscriptions

[heyrict]

Reopen of #1355 .

To inspect whether it is good to use event-based subscriptions or live queries, I've gone through many articles and found this comment best describes the current state of live queries and event-based subscriptions.

There are still use cases of event-based subscriptions, especially when performance count more than reliability (take an example of twitter like subscription)

Another thing is that, most subscriptions in the current graphql ecosystem use event-based ones. It will be hacky to convert an existing project with subscriptions to hasura.

The hacky part of using live query for event-based subscriptions is that

1. It will be difficult or at least not straight-forward to detect row deletion. Tracking node deletion using subscriptions #1201
2. It will yield unnecessary initial results.
3. Tracking update or create events need extra effort (adding updated_at columns to sort with, etc.)

cc: @coco98

[heyrict]

Here is a cleaner workaround to have event-based subscription, though it needs another graphql server instance to expose the subscription API.
See heyrict/hasura-trigger-subscription-server.

[0x777]

@heyrict One of the biggest challenges/decisions with event based subscriptions is how does the client recover from loss of events? Does the server store what events are delivered to clients? This involves maintaining state at the server (using something like a message queue maybe?) and becomes even more complicated when you bring in horizontal scalability of the subscriptions server.

We support live queries because they are easy to reason about, very expressive, extremely performant and the server need not store any state (in our implementation).

[heyrict]

@0x777 Well, I like hasura's performant design and don't think bundling another layer inside to maintain state is a good choice.
However, there are still use cases for event based subscriptions and I've seen people using hasura as a remote schema and use custom logic to handle subscriptions. May be it is reasonable to inform users about the possible workarounds in the docs?

[bkniffler]

One thing I've found is, it is very hard to write a mutation that would optimistically update a subscription with apollo, or maybe even impossible, since apollo doesn't seem to have build in mechanisms to automatically update subscriptions (using optimisticResponse) and there is no possibility to read/write subscriptions from/to cache. This must be due to the fact that, as @heyrict states, most implementations and even apollo docs use event based subscriptions instead of live queries.

Would love to see some workaround for live-queries + optimistic response, I couldn't manage to get this combination to work with apollo.

[heyrict]

@bkniffler I am not sure if I have understand live-queries + optimistic response properly. You CAN use live queries with optimistic response.

If you are using

<Subscription subscription={SOME_LIVE_QUERY}>
   {this.renderLiveQuery}
</Subscription>

Then the subsciption data will always reflect what SOME_LIVE_QUERY queries to (though with some latency), and is not designed to be modified.

If you want to use optimistic response to update the live query, you should store query data in local cache with a Query component.
In this case, you have to write a custom update function in subscribeToMore as apollo will not automatically merge the live query with the cache.

<Query query={INITIAL_QUERY}>
  {params => <QueryRenderer {...params} />}
</Query>

class QueryRenderer extends Component {
  componentDidMount() {
    this.unsubscribe = this.props.subscribeToMore({
      query: SOME_LIVE_QUERY,
      update: (...) => {/* Logic to update cache here */},
      ...
    });
  }
  ...
}

[RodolfoSilva]

I believe if hasura add support to redis, this feature will be possible.@0x777

[mitar]

What has this with Redis?

[barbalex]

@heyrict One of the biggest challenges/decisions with event based subscriptions is how does the client recover from loss of events? Does the server store what events are delivered to clients? This involves maintaining state at the server (using something like a message queue maybe?) and becomes even more complicated when you bring in horizontal scalability of the subscriptions server.

We support live queries because they are easy to reason about, very expressive, extremely performant and the server need not store any state (in our implementation).

@0x777

The problem with live queries is that they always deliver all the information.

Say I have a table with 10'000 records. One of them changes.
So the live query fires. Now my application needs to find out which record changed in order to update it client side. So it has to compare all 10'000 records!

If event based subscriptions existed, the client could:

• at startup: do the expensive comparison only once to ensure all data is up to date (actually the client only needs to query data changed since the last startup)
• start the event based subscription
• cheaply update only the changed records when events happen

This seems so obvious I am sure I must have missed something essential. What have I missed?

[nolandg]

I posted what might be a slightly simpler work-around for some scenarios here: apollographql/apollo-client#5267

[maxpain]

Any updates on this?

[marcusrohden]

hi guys, any updates on this? would be amazing to have the ability to receive the initial data + only the changed data in future updates.

[pleclech]

@marcusrohden What you want is probably looking at the streaming API: https://www.youtube.com/watch?v=C1TDDlXPSmI

[maxpain]

Any updates?

[vincentjames501]

Also wondering if this has been explored anymore!