Broadcasting options #2

Open
palkan opened this Issue Dec 4, 2016 · 11 comments

Comments

Projects
None yet
7 participants
@palkan
Member

palkan commented Dec 4, 2016

Currently, AnyCable requires Redis as a dependency for the only one thing: sending messages from the application to the WebSocket server for broadcastings (ActionCable.server.broadcast ...).

The idea is to provide more options for the users and thus not to require Redis for everyone.

TODO adapters:

Other possible options:

  • UDP ?
  • ...

@palkan palkan added the discussion label Dec 4, 2016

@kobaltz

This comment has been minimized.

Show comment
Hide comment
@kobaltz

kobaltz Dec 21, 2016

Personally, I like having Redis as a dependency as most of my current applications use two Redis instances; persistent store and volatile. However, I'd always be open to something different if the maintainability/performance was the same or better. I would caution against PostgreSQL as a dependency as that would be more limiting than having a Redis dependency. While I do use PostgreSQL in some applications, we have to use other databases in other cases.

kobaltz commented Dec 21, 2016

Personally, I like having Redis as a dependency as most of my current applications use two Redis instances; persistent store and volatile. However, I'd always be open to something different if the maintainability/performance was the same or better. I would caution against PostgreSQL as a dependency as that would be more limiting than having a Redis dependency. While I do use PostgreSQL in some applications, we have to use other databases in other cases.

@maurcarvalho

This comment has been minimized.

Show comment
Hide comment
@maurcarvalho

maurcarvalho Dec 21, 2016

HTTP REST seems like an "out of external dependency" way to go.

@palkan can you say a little more about how do you imagine it?
will it be something like a http handler between WS and ActionCable.server.broadcast ...

HTTP REST seems like an "out of external dependency" way to go.

@palkan can you say a little more about how do you imagine it?
will it be something like a http handler between WS and ActionCable.server.broadcast ...

@palkan

This comment has been minimized.

Show comment
Hide comment
@palkan

palkan Dec 21, 2016

Member

@kobaltz The idea is to avoid additional dependency if it's possible. Most Rails applications already use Redis, so they are likely to be OK with it (and that's why for now Redis is the only option, btw).

But there are applications which don't use it. And that's the main reason why we're looking for alternatives.

There can also be one more reason: performance and scalability. For i-dunno-how-much-large apps, it would be better to use RabbitMQ or even Kafka.

We do not have to provide all possible options but an abstraction to build them (as separate gems).
On the other hand, to develop a good abstraction it would be useful to know these alternatives beforehand.

P.S. As you probably remember, Action Cable initially also depended on Redis, the community didn't like it.

Member

palkan commented Dec 21, 2016

@kobaltz The idea is to avoid additional dependency if it's possible. Most Rails applications already use Redis, so they are likely to be OK with it (and that's why for now Redis is the only option, btw).

But there are applications which don't use it. And that's the main reason why we're looking for alternatives.

There can also be one more reason: performance and scalability. For i-dunno-how-much-large apps, it would be better to use RabbitMQ or even Kafka.

We do not have to provide all possible options but an abstraction to build them (as separate gems).
On the other hand, to develop a good abstraction it would be useful to know these alternatives beforehand.

P.S. As you probably remember, Action Cable initially also depended on Redis, the community didn't like it.

@palkan

This comment has been minimized.

Show comment
Hide comment
@palkan

palkan Dec 21, 2016

Member

@maurcarvalho Sure.

Agree, that's the easiest option (from the development point of view).

Assume that our WebSocket server implements HTTP endpoint to receive broadcast messages. Then we can write our #broadcast method like this:

def broadcast(channel, payload)
  Net::HTTP.post_form(AnyCable.config.http_endpoint, { "channel" => channel, "payload" => payload })
end

Of course, it's a very straightforward solution. It would be great to re-use connection ('keep-alive') of even have a connection pool, send requests asynchronously and deal with errors.
Maybe, we should put it to ActiveJob out-of-box.

Not so easy)

Member

palkan commented Dec 21, 2016

@maurcarvalho Sure.

Agree, that's the easiest option (from the development point of view).

Assume that our WebSocket server implements HTTP endpoint to receive broadcast messages. Then we can write our #broadcast method like this:

def broadcast(channel, payload)
  Net::HTTP.post_form(AnyCable.config.http_endpoint, { "channel" => channel, "payload" => payload })
end

Of course, it's a very straightforward solution. It would be great to re-use connection ('keep-alive') of even have a connection pool, send requests asynchronously and deal with errors.
Maybe, we should put it to ActiveJob out-of-box.

Not so easy)

@AnthonySuper

This comment has been minimized.

Show comment
Hide comment
@AnthonySuper

AnthonySuper Dec 24, 2016

If you're already using GRPC, why not also use it to deliver messages to the server?

It can be something like:

service AnycableServer {
    rpc Broadcast (BroadcastingMesssage) returns (google.protobuf.Empty) {}
}

message Message {
    string channel = 1;
    string payload = 2;
}

Now, this would make a server a bit harder to implement, which is something we need to consider.

If you're already using GRPC, why not also use it to deliver messages to the server?

It can be something like:

service AnycableServer {
    rpc Broadcast (BroadcastingMesssage) returns (google.protobuf.Empty) {}
}

message Message {
    string channel = 1;
    string payload = 2;
}

Now, this would make a server a bit harder to implement, which is something we need to consider.

@palkan

This comment has been minimized.

Show comment
Hide comment
@palkan

palkan Dec 26, 2016

Member

If you're already using GRPC, why not also use it to deliver messages to the server?

That's the reason:

... this would make a server a bit harder to implement

We have to run GRPC server as a part of our WebSocket server, which is not available yet for all languages (e.g. for Erlang).

And another problem: consider clustered configuration – several WebSocket servers, several application servers. Then we need a GRPC balancer – yet another problem.

Member

palkan commented Dec 26, 2016

If you're already using GRPC, why not also use it to deliver messages to the server?

That's the reason:

... this would make a server a bit harder to implement

We have to run GRPC server as a part of our WebSocket server, which is not available yet for all languages (e.g. for Erlang).

And another problem: consider clustered configuration – several WebSocket servers, several application servers. Then we need a GRPC balancer – yet another problem.

@AnthonySuper

This comment has been minimized.

Show comment
Hide comment
@AnthonySuper

AnthonySuper Dec 26, 2016

Okay, gotcha. I didn't even consider the load balancing part, which is a pretty big issue now that I think about it.

An HTTP solution seems like it will likely be the best for now, in that case.

Okay, gotcha. I didn't even consider the load balancing part, which is a pretty big issue now that I think about it.

An HTTP solution seems like it will likely be the best for now, in that case.

@palkan palkan referenced this issue in anycable/anycable-rails Apr 18, 2017

Closed

failover support #3

@pbrumm

This comment has been minimized.

Show comment
Hide comment
@pbrumm

pbrumm Apr 18, 2017

I think having nsq would be very interesting for the message passing.
http://nsq.io/components/nsqd.html

it enables the actioncable component to just put the message into a queue, and then each instance of the websocket server could register a channel on that queue with a name from a config file. This would allow multiple websocket servers to receive messages without concern for maintaining an active connection with the rails app.

the rails app could also be modified to just be reading messages from a queue with a single channel.

nsq has client libraries in many languages and is written in go.

pbrumm commented Apr 18, 2017

I think having nsq would be very interesting for the message passing.
http://nsq.io/components/nsqd.html

it enables the actioncable component to just put the message into a queue, and then each instance of the websocket server could register a channel on that queue with a name from a config file. This would allow multiple websocket servers to receive messages without concern for maintaining an active connection with the rails app.

the rails app could also be modified to just be reading messages from a queue with a single channel.

nsq has client libraries in many languages and is written in go.

@slayer

This comment has been minimized.

Show comment
Hide comment
@slayer

slayer Jun 10, 2017

What about MQTT ?

slayer commented Jun 10, 2017

What about MQTT ?

@palkan

This comment has been minimized.

Show comment
Hide comment
@palkan

palkan Jun 10, 2017

Member

@slayer

What about MQTT ?

Although, MQTT-based broadcasting is possible, I think, it would too complicated for our use case. The main (IMO) feature of MQTT – quality of service – doesn't make sense in our case: if a WebSocket server is down and doesn't receive broadcast messages (through HTTP/Redis/queue), it's likely not to handle client connections too.

Broadcast messages a ephemeral from the WS server point of view. At least for now.

Member

palkan commented Jun 10, 2017

@slayer

What about MQTT ?

Although, MQTT-based broadcasting is possible, I think, it would too complicated for our use case. The main (IMO) feature of MQTT – quality of service – doesn't make sense in our case: if a WebSocket server is down and doesn't receive broadcast messages (through HTTP/Redis/queue), it's likely not to handle client connections too.

Broadcast messages a ephemeral from the WS server point of view. At least for now.

@jtoy

This comment has been minimized.

Show comment
Hide comment
@jtoy

jtoy Jul 13, 2018

love the idea of postures pub/sub

jtoy commented Jul 13, 2018

love the idea of postures pub/sub

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment