/subcription - web sockets or HTTP-SSE ?

[lforesta]

We have started implementing the /subscription endpoint using web sockets as detailed in the subscriptions api here.

Instead of using web sockets, it is also possible to use HTTP Server Sent Events as suggested by @selloween . See a comparison of the two methods here

Having only one protocol in the API has its advantages in terms of implementation, so I was wondering what is the reason for choosing web sockets instead of HTTP-SSE?

@m-mohr mentioned that browsers may allow only a handful of simultaneous HTTP connections, so this may in fact a reason to go with web sockets.

Did anyone already work a bit on this service (beside Matthias)? Any thoughts?

[aljacob]

Here is the link again, for me the on from Luca didn't work: do you really need websockets. There is also a longer blog entry on microsofts blogs here: micrsofts blog on websockets. I personally have no strong opinion on that as of know, since I didn't implement neither. Not sure how important it is for us to support web browser like (IE, EDGE). Most of our interaction is foreseen to be happening in client API's anyways, or how do you see that?

[jankovicgd]

One other issue we might face is that we are inside of TU Wien (TUNET) network which restricts certain protocols and ports. With HTTP-SSE we might work around any available restrictions.

[selloween]

I'm not involved in development and not a developer. Using HTTP-SSE was just a suggestion regarding server to client notifications. I know websockets for bidirectional communication used in chat, online games and social feeds. If it runs on our infrastructure without constraints, its for development to decide - I'm fine either way as long as its tested before going into production.

[m-mohr]

Okay, I wrote quite a lot here but the browser crashed just before submitting it. So here's a short version:

• SSE has advantages because it's easier to implement (Auth, Handshake) overall, but has limited library support (no R support yet?; Edge doesn't implement it yet, but Microsoft will switch to Chromium soon, so shouldn't be a big deal for the Web Editor in the future @aljacob ). SSE ist harder to document as part of a single endpoint - also: not supported in OpenAPI.
• WS have advantages when it comes to subscribe/unscubsribe and allow full-duplex, but both is probably not required for our use cases at the moment. WS has better library support.
• Neutral: WS and SSE both use ports 80/443 for communication, so I don't see a big problem with network restrictions. @jankovicgd

Implementation is probably much easier for SSE, expect for the R client. That's the most tempting point for me to switch. Overall WS seems to have slight advantages if we compare pros and cons. Not sure yet, maybe we just need to try it out? I could come up with a SSE specification easily as messages would be the same and it's basically just copy&paste. We could just try it...

Some things to consider/check:

• We probably only want to have a single SSE connection per user as otherwise we may reach the limit of parallel HTTP connections (around 6). Is this easy to implement when multiple microservices want to send messages?
• How well do they both work for long-term subscriptions, e.g. getting notifications about new datasets? We'll also have server-server communication, I guess. It's not only server-client.

[flahn]

I haven't looked into SSE vs. Websockets yet, but I think it should not be that difficult to implement it for the R client. I can try to integrate either a C/C++ library or I can use an internal HTML / JS browser from R. The functionality for the subscription boils down to receiving and visualising messages from the server, so this should be possible somehow.

[jankovicgd]

I am voicing my concerns. I am by no means an expert, just attempting to contribute, note the usage of the word might. Seems like practically the complexity increases and there might even be no need. Not saying it doesn't make sense since Stack Exchange provides a REST API along with a WebSocket API. Leaving links for reference.

1. Establishing a connection for up to 30 seconds and allowing messages might cause bandwidth problems. This might also connect to the point 2.
   • Don’t allow anyone to establish a connection and then wait for them to authenticate over the websocket itself
2. Auth for backends might need to be handled differently than currently proposed.
   • https://devcenter.heroku.com/articles/websocket-security#authentication-authorization
   • Passing a header with browser js is not possible, although I have managed to pass it with python
3. Websocket is not necessarily a good protocol if neither of the endpoints is an actual browser. Websockets suffer a lot under the restrictions and implementation details that were forced upon the protocol to make it work with existing HTTP infrastructure.
4. Clients would have to account for IP addresses changing and connection dropping.

All of the links I come across say Websockets VS REST/HTTP. Can someone give an openeo-centric example use case with both the API and the notification service? Would clients really need to keep open connections for so long? Would there need to be separate clients that consume REST and Websockets? Would it make sense to separate a backend into the API and WSS?

[mkadunc]

My 2¢ — if I had a choice i'd do away with notifications and just implement polling on the client.

Second best IMO would be to go with SSE, because it's more high-level and directly targets our use case (while websockets is totally generic and we'd need to invent something on top).

[m-mohr]

We decided on the 3rd year planning to go with polling for now in the Core API.

Debugging and error messages will be written to a log, which still has to be added to the API, see #214.

As I'd like to experiment with the debug process I'll probably make the WebSocket specification available as an extension.

Issue is only open for making it an extension. Can be closed afterwards.

[m-mohr]

Closed for now as the WebSocket-based Subscriptions API won't be part of 1.0. We may revisit this in the future.

The code for the Subscriptions API is still available in the feature/subscriptions-api branch.