Replace long polling to receive events with Server Sent Events (SSE)

[stalniy]

Description:

Server Sent Events(SSE) is better than long polling because it creates only 1 persisted connection to server and keeps it open "forever". This technology was specifically designed to overcome challenges related to short/long polling.
SSE is built on top of HTTP protocol, so also can be used in communication between servers. Also SSE is more compliant with existing IT infrastructure like (Load Balancer, Firewall, etc), unlike WebSockets which can be blocked by some firewall.

Currently all major web browsers supports SSE (for those which don't support it's easy to do fallback to long/short polling). There are some cool features which browsers support out of the box:

• automatic reconnection
• remember lastEventId and send it automatically on reconnection.

Check this for reference:

• https://www.html5rocks.com/en/tutorials/eventsource/basics/
• https://stackoverflow.com/questions/9397528/server-sent-events-vs-polling
• https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events
• supported browsers: https://caniuse.com/#feat=eventsource

[richvdh]

this would need to be a matrix protocol change, so moving to matrix-doc.

[richvdh]

or not, since github is giving 500 errors right now

[turt2live]

moved

[stalniy]

What can I do to push this forward? Do I need to create a proposal to protocol change as described here?

[turt2live]

Yup. That'd be the first steps

[uhoreg]

I've thought about this a bit before, so I'll dump my thoughts here. (I don't have time to work on it right now.)

• create a new endpoint to do sync over SSE (e.g. /sync/stream)
  • used to get the same data as continually calling /sync with the same parameters
  • accepts the same parameters as /sync, except without since or timeout
  • instead of using the since query parameter, the next batch token will be passed through the Last-Event-ID header.
• each event will have the same format as what /sync returns. The id of each event will be the next_batch token
  • the server sends events in exactly the same way that it would send responses to /sync calls with the since parameter set to the previous next_batch

Using SSE in this way means that existing code can be reused for SSE

[stalniy]

Created proposal for the change

[ell1e]

Isn't there a websocket draft already? https://github.com/matrix-org/matrix-spec/blob/main/attic/drafts/websockets.rst Wouldn't that one already serve the same purpose?

[stalniy]

The important thing is that event stream is just an HTTP request and it should be easier to use it from native clients (iOS, Android) and for cross server communication

[ell1e]

Ah, so it's just easier to implement. Fair enough, I guess that is as good enough of a reason as any, in the end it doesn't really matter after all what approach is used. So I assume if this goes through that websocket draft will go unused? (It'd be a bit weird to attempt to implement both)

[ell1e]

For what it's worth, this is somewhat not only blocking F-Droid/non-gapps Android matrix clients from decent battery life, but also those on Linux mobiles like the PinePhone/PineTab/Librem 5, or new ARM64 laptops like repurposed Chromebooks with Linux or the Pinebook. (Since on those platforms, there simply isn't any Firebase or similar available either. Well, Ubuntu Touch has it, but none of all of the other linux variants.)