Skip to content
This is a ✨special✨ repository containing the organization level discussions for anycable. Everything posted here will also be visible at the organization level.

Broadcasting options #2

palkan started this conversation in Ideas
Broadcasting options #2
Dec 4, 2016 · 19 comments · 3 replies

Currently, AnyCable requires Redis as a dependency for the only one thing: sending messages from the application to the WebSocket server for broadcastings (ActionCable.server.broadcast ...).

The idea is to provide more options for the users and thus not to require Redis for everyone.

TODO adapters:

Other possible options:

  • UDP ?
  • ...

Replies

19 comments
·
3 replies

Personally, I like having Redis as a dependency as most of my current applications use two Redis instances; persistent store and volatile. However, I'd always be open to something different if the maintainability/performance was the same or better. I would caution against PostgreSQL as a dependency as that would be more limiting than having a Redis dependency. While I do use PostgreSQL in some applications, we have to use other databases in other cases.

0 replies

HTTP REST seems like an "out of external dependency" way to go.

@palkan can you say a little more about how do you imagine it?
will it be something like a http handler between WS and ActionCable.server.broadcast ...

0 replies

palkan
Dec 21, 2016
Maintainer Author

@kobaltz The idea is to avoid additional dependency if it's possible. Most Rails applications already use Redis, so they are likely to be OK with it (and that's why for now Redis is the only option, btw).

But there are applications which don't use it. And that's the main reason why we're looking for alternatives.

There can also be one more reason: performance and scalability. For i-dunno-how-much-large apps, it would be better to use RabbitMQ or even Kafka.

We do not have to provide all possible options but an abstraction to build them (as separate gems).
On the other hand, to develop a good abstraction it would be useful to know these alternatives beforehand.

P.S. As you probably remember, Action Cable initially also depended on Redis, the community didn't like it.

0 replies

palkan
Dec 21, 2016
Maintainer Author

@maurcarvalho Sure.

Agree, that's the easiest option (from the development point of view).

Assume that our WebSocket server implements HTTP endpoint to receive broadcast messages. Then we can write our #broadcast method like this:

def broadcast(channel, payload)
  Net::HTTP.post_form(AnyCable.config.http_endpoint, { "channel" => channel, "payload" => payload })
end

Of course, it's a very straightforward solution. It would be great to re-use connection ('keep-alive') of even have a connection pool, send requests asynchronously and deal with errors.
Maybe, we should put it to ActiveJob out-of-box.

Not so easy)

0 replies

If you're already using GRPC, why not also use it to deliver messages to the server?

It can be something like:

service AnycableServer {
    rpc Broadcast (BroadcastingMesssage) returns (google.protobuf.Empty) {}
}

message Message {
    string channel = 1;
    string payload = 2;
}

Now, this would make a server a bit harder to implement, which is something we need to consider.

0 replies

palkan
Dec 26, 2016
Maintainer Author

If you're already using GRPC, why not also use it to deliver messages to the server?

That's the reason:

... this would make a server a bit harder to implement

We have to run GRPC server as a part of our WebSocket server, which is not available yet for all languages (e.g. for Erlang).

And another problem: consider clustered configuration – several WebSocket servers, several application servers. Then we need a GRPC balancer – yet another problem.

0 replies

Okay, gotcha. I didn't even consider the load balancing part, which is a pretty big issue now that I think about it.

An HTTP solution seems like it will likely be the best for now, in that case.

0 replies

I think having nsq would be very interesting for the message passing.
http://nsq.io/components/nsqd.html

it enables the actioncable component to just put the message into a queue, and then each instance of the websocket server could register a channel on that queue with a name from a config file. This would allow multiple websocket servers to receive messages without concern for maintaining an active connection with the rails app.

the rails app could also be modified to just be reading messages from a queue with a single channel.

nsq has client libraries in many languages and is written in go.

0 replies

What about MQTT ?

0 replies

palkan
Jun 10, 2017
Maintainer Author

@slayer

What about MQTT ?

Although, MQTT-based broadcasting is possible, I think, it would too complicated for our use case. The main (IMO) feature of MQTT – quality of service – doesn't make sense in our case: if a WebSocket server is down and doesn't receive broadcast messages (through HTTP/Redis/queue), it's likely not to handle client connections too.

Broadcast messages a ephemeral from the WS server point of view. At least for now.

0 replies

love the idea of postgres pub/sub

0 replies

Re:

HTTP REST – great for development/testing,

Nice! 👍

small projects with the only one websocket server instance

Just trying to understand more, if understand correctly, the main advantage of AnyCable is: to write a high-performant real-time application using Ruby as a language for implementing a business-logic.

Wouldn't depending on a HTTP transport medium null the performance benefits that is gained by AnyCable specifically for small projects?

I think it is also subjective of how small is a small project, but wondering to get more of your thoughts about it.

0 replies

palkan
Jan 23, 2019
Maintainer Author

Wouldn't depending on a HTTP transport medium null the performance benefits that is gained by AnyCable specifically for small projects?

Yep, from the performance point of view there is no a huge benefit.
But you can still have benefits from the resources usage point of view.

how small is a small project

Yeah, that's a good question.
We should think about the number of simultaneous connections (peak and average) and the message rate/payload size.
I think, the threshold to start thinking about AnyCable (instead of just Action Cable) is somewhere between 500 and 1000 connections on average or 5k-10k during peak hours.

0 replies
0 replies

palkan
Jan 23, 2019
Maintainer Author

What’s the largest known anycable project?

The largest project I'm aware of (and I'm not aware of the most of them, unfortunately) had ~10k concurrent connections (that's equipe.com, one year ago); they handled this with 4 1x dynos on Heroku (before switching to AnyCable they had 20 2x dynos for ActionCable).

We are testing now if we can support 60k concurrent connections.

Could you please share some information on your use case?

0 replies

@palkan Thanks for the context! – I see you abstracted the broadcasting adapters already, I will give it a try with adding one of the broadcasting adapter options you listed above, I will tag you in a PR ☺️

0 replies

palkan
Jan 26, 2019
Maintainer Author

@jtoy
Could you please share some information on your use case? I’ve added an issue to collect the feedback:
#68

0 replies

hey we like the idea to add a rabbitmq adapter. Could you explain a bit more detailed which steps would be necessary to achive this? Is it right to also adapt the anycable-go app?

1 reply
@palkan

palkan Jan 15, 2021
Maintainer Author

Yep, pub/sub adapter should be added to both RPC and WS server implementations.

We use a single stream/queue/channel to deliver messages from RPC to WS.
RPC server acts as publisher: it pushes a JSON-encoded command. Pubsub connection is initialized lazily in this case (during the first #broadcast call).

WS server (anycable-go) acts as subscriber: subscription is initialized on server start, messages are received, deserialized and passed to the app.

Hi @palkan, Anycable is an amazing project! I'm trying to build a prototype for my work. I read through anycable-go's source code. It seems anycable-go only subscribes to one redis pubsub channel, which is anycable. According to official Actioncable guide, Actioncable creates multiple redis pubsub channels. Could you please confirm this is a difference between Anycable and Actioncable? If it is the case, could you please share more details on that? I have been reading source code and searching answers online. But, there's very little information about how Anycable/Actioncable uses redis channels. Thank you so much!

We are working for one of the largest tech company who is based on Ruby on Rails. Right now, we are building a concept proofing prototype using Anycable. Please help us, so we can persuade the leadership to invest resource on it. 🙏

2 replies
@palkan

palkan Jan 28, 2021
Maintainer Author

Could you please confirm this is a difference between Anycable and Actioncable? If it is the case, could you please share more details on that?

Yes, AnyCable uses only a single Redis pub/sub channel. Unlike Action Cable, anycable-go manages the actual subscriptions by itself (see hub.go), we only need a single channel to get broadcasts from web apps to a WS server, which performs the actual retransmission. Check out https://docs.anycable.io/#/v1/misc/how_to_anycable_server

@ztsuperman

Thank you so much for your quick answer! 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Ideas
Labels
None yet
10 participants
Converted from issue