New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Load balance among clients #778

Closed
pemmasanikrishna opened this Issue Jun 4, 2018 · 18 comments

Comments

Projects
None yet
3 participants
@pemmasanikrishna

pemmasanikrishna commented Jun 4, 2018

A new feature for client side load balancing would be helpful for having multiple clients listening to the channel, but avoiding duplication.

@sbordet

This comment has been minimized.

Member

sbordet commented Jun 4, 2018

Can you detail exactly why would you need this feature and how exactly it would work ?

@pemmasanikrishna

This comment has been minimized.

pemmasanikrishna commented Jun 4, 2018

I would like to scale number of clients horizontally, so that there wouldnt be a bottle neck between the cometd and the client. If there is a fast publisher, and the client is slow, it would be really handy to have the clients scaled horizontally.

So, with this feature, we can configure such a way that though multiple clients are listening, only one client receives a particular message and others dont in this mode, may be we can use round robin load balance as well among the clients.

@sbordet

This comment has been minimized.

Member

sbordet commented Jun 4, 2018

I'm not sure I understand.

If you want to scale on a single machine, it would be enough to handle the messages received by a single client in a different thread - that would be the same as having multiple clients on the same machine but without the complication.

If you want to scale on multiple machines, you still need a single load balancer to load balance the load across multiple clients each on its own machine, so you still have a single component that needs to scale - and you can write this at the application level.

Alternatively, it's trivial and probably more efficient to round robin on the server side.

It's still not clear to me what exactly the problem is, whether you have actually measured a difference in performance (and of what measure) and how would your solution work in details.

@pemmasanikrishna

This comment has been minimized.

pemmasanikrishna commented Jun 5, 2018

Yes, thats true. We can have a load balancer , but still it would route all the messages sent to the channel to all the machines, but in intelligent scenario it should send only to one of the machines.
Suppose there are 3 machines that are subscribed to a channel "/foo", Then when 4 messages are sent consecutively to that channel, machine1 should receive the 1st message, machine 2 the second, m3 the 3rd and m1 the 4th. But this should be done at the client level or the server level.

@sbordet

This comment has been minimized.

Member

sbordet commented Jun 5, 2018

@pemmasanikrishna what if one of the machines goes down ?

I think that it's way better to have this done on the server side, in the application rather than in the library.
It would be trivial for you to know the N clients, and then use ServerSession.deliver() to round robin (or use any other algorithm) the message delivery. The acknowledgement extension will guarantee server-to-client message delivery.

@pemmasanikrishna

This comment has been minimized.

pemmasanikrishna commented Jun 5, 2018

may be, even server side sounds good. If one of the machines go down, it shouldn't affect the behavior of the system, nor should the data be lost. Do we have already anything existing at the server side?

@pemmasanikrishna pemmasanikrishna changed the title from Client side load balance to Load balance among clients Jun 5, 2018

@sbordet

This comment has been minimized.

Member

sbordet commented Jun 5, 2018

CometD already has all the primitives to do this yourself, using ServerSession.deliver() and the acknowledgement extension.

@pemmasanikrishna

This comment has been minimized.

pemmasanikrishna commented Jun 6, 2018

But what if we don't have control over the server? May be like in the case of Salesforce platform events.

@sbordet

This comment has been minimized.

Member

sbordet commented Jun 6, 2018

If you don't have control over the server, you are back at having a single "load balancer" client that needs to republish to other clients on other machines, and you have just moved the problem to a different spot.

Now this single client emits events for other clients, so for those other clients it is a server, and there you implement the logic you want - you have implemented a proxy.

Basically it is this:

   proxy
   +----------------+
   | proxy client <-|------> server
   |    -  -  -     |
   | proxy server   |
   |  ^        ^    |
   +--|--------|----+
      |        |
      v        v
   client1  client2

The server emits the events, they are received by the proxy client, which republishes them to the proxy server with a round robin logic, so the clients get the messages with the logic you want.

Now you have to make the proxy scale well enough to keep up with the server - from the CometD point of view there should be no problems.

This proxy is not already available in CometD, but it's fairly easy to write as I suggested in previous messages.
If you need help, see this.

@pemmasanikrishna

This comment has been minimized.

pemmasanikrishna commented Jun 7, 2018

Yeah, i do agree that its easy to implement, but was wondering what advantage do we get by shifting problem to a different spot. Instead if the java client has a mechanism at least to query the server and get all the active clients subscribed for a particular channel, that would really help.

@sbordet

This comment has been minimized.

Member

sbordet commented Jun 7, 2018

I don't see how it would help, can you detail?

@pemmasanikrishna

This comment has been minimized.

pemmasanikrishna commented Jun 12, 2018

By retrieving the active clients that are currently listening to a particular channel, clients can have their own implementation of load balancing if they don't have control over server.

@sbordet

This comment has been minimized.

Member

sbordet commented Jun 12, 2018

@pemmasanikrishna feel free to implement your solution, I'll have a look if you'd like.

@Diagoras

This comment has been minimized.

Diagoras commented Jun 13, 2018

@sbordet Also joining in on this ask. While you're correct that a proxy can handle distributing the messages to workers, it's still a single point of failure. When using the Salesforce Streaming API, we'd generally prefer to only have Salesforce deal with the overhead and complexity of maintaining a stable CometD server and only have to deal with the client end ourselves.

I think this will likely require a change in the server, to ensure that duplicate messages aren't pushed to multiple clients and to make clear which clients are associated. You'd know better than me, though.

@sbordet

This comment has been minimized.

Member

sbordet commented Jun 13, 2018

@Diagoras yes you are right that it would be a server job to round robin the messages to different clients. We discussed this in a previous comment, but being the Salesforce server it cannot be easily changed (and it's not a trivial task on the server either).

So a proxy solution would work and yes, it would still be a single point of failure. This can be mitigated in various ways (e.g. master/slave proxies), but it's quite some work.

@sbordet sbordet closed this Sep 4, 2018

@sbordet sbordet added this to the 3.1.5 milestone Sep 4, 2018

@Diagoras

This comment has been minimized.

Diagoras commented Sep 4, 2018

@sbordet I see that this issue was closed and added to 3.1.5. Does that mean a PR to address the issue has been merged? If so, would you mind linking it to this issue?

@sbordet

This comment has been minimized.

Member

sbordet commented Sep 4, 2018

There is no PR, this was not an bug but only a discussion of a particular scenario that is fixable on the server application.

One of the ideas proposed was, IMHO, not working (or I could not see how it could possibly work). I asked for details, but none.

Your comment about fixing it on the SalesForce server is right, but you have to ask them.

So from my point of view this discussion is closed and there is nothing CometD needs to do.

If that's not the case, then I need details of what CometD should do, because the discussion so far is all about application-specific behavior that can already be coded with the current CometD version.

@Diagoras

This comment has been minimized.

Diagoras commented Sep 5, 2018

Understood. It sounds like you think this isn't the responsibility of the Bayeux protocol and CometD server, but instead should be layered on top? Maybe as an extension?

If so, that makes sense to me. Closing this out sounds good.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment