Technical questions about horizontal scaling

[vincentfretin]

Hello the Hubs team,

After looking at the documentation https://hubs.mozilla.com/docs/hubs-cloud-aws-architecture.html and the code at https://github.com/mozilla/hubs-ops
I understand the following (please correct me if I'm wrong), the app servers is mainly reticulum, a phoenix/elixir app running on erlang VM.
The streaming servers is the janus sfu or now mediasoup dialog sfu (nodejs process apparently? https://github.com/mozilla/dialog)

In a 2x2 server configuration (2 app servers and 2 stream servers), does the 2 app servers talk to each other? Does the 2 stream servers talk to each other?
As far as I understand, for chat in a room, the data goes through the phoenix websocket.
As far as I can see in https://github.com/mozilla/hubs-ops/blob/master/cloudformation/stack.yaml, the aws load balancer can just be a dns round robin between app server 1 and app server 2. So imagine that if Alice goes to app server 1, and Bob goes to app server 2, how does the two communicate with each other? I read some documentation about elixir that an app can be scaled horizontally between several nodes, is this what's going on? How is this configured on aws?
I read that in other tech like nodejs with socket.io that you will need an additional redis to communicate between the nodejs process hosted on each node.

For janus I read that the publisher (the one sending the audio or video) and subscribers (listening to the audio, seeing a screen share) needs to be on the same janus instance. And one way for horizontal scaling is to be sure that the people in the same room goes to the same janus instance.
From the Hubs code, it doesn't seem you're doing that.
I only found out https://github.com/mozilla/reticulum/tree/master/lib/ret/load_balancing that suggests that the janus instance returned is the one having the less ccu (number of sessions from janus admin). How does it work now for mediasoup dialog? Can you point me to the relevant part of the code?
If I understand this correctly, based on what I read on the janus mailing-list, if Alice and Bob go through a different janus instance, the WebRTC connection between them can't be established. I'm wondering if it's not what happen for some people that can see each other avatar (data from phoenix presence) but can't hear each other?

If you can answer some of those technical questions for my own curiosity, I will appreciate it. @gfodor may be the person that know this best.

[vincentfretin]

I'm currently learning Elixir and Phoenix. I just finished https://pragmaticstudio.com/courses/elixir and next are
https://pragmaticstudio.com/courses/unpacked-multi-player-bingo-with-elixir-phoenix-vue-elm and https://pragmaticstudio.com/courses/unpacked-full-stack-graphql-with-absinthe-phoenix-react
I am pleased with those courses so far. If you want to learn Elixir and Phoenix, I highly recommend.
There are other courses listed there https://hexdocs.pm/phoenix/community.html

Looking again at the reticulum code, I see that Phoenix.Presence is configured to use pubsub_server: Ret.PubSub (https://github.com/mozilla/reticulum/blob/60f5ff32ec564a0e5b9933f11bd25f295ca94165/lib/ret_web/channels/presence.ex#L2-L4)
and Ret.PubSub is configured to use Phoenix.PubSub.PG2 (https://github.com/mozilla/reticulum/blob/22c2e4b3cd5fdbce1b5f8d8be6cf02abeca2b206/config/prod.exs#L32)
From the documentation https://hexdocs.pm/phoenix_pubsub/Phoenix.PubSub.html this adapter uses Distributed Elixir, directly exchanging notifications between servers. https://elixirschool.com/en/lessons/advanced/otp-distribution/

[vincentfretin]

I found the answer when reading the new graphql documentation https://github.com/mozilla/reticulum/pull/399/files#diff-2ecdd4fbde0a5a21f68a9d0bbe630e0dR73-R74
So you indeed assign the same node to all participants of a room, great :-)
When you create a room you assign the host to the room https://github.com/mozilla/reticulum/blob/0cde188f38f6ac4677ae261da178f65e6ea5f72d/lib/ret/hub.ex#L89
You ensure that the host is not nil and is alive, otherwise you assign another host https://github.com/mozilla/reticulum/blob/0cde188f38f6ac4677ae261da178f65e6ea5f72d/lib/ret/hub.ex#L371-L385
The websocket returns this host https://github.com/mozilla/reticulum/blob/0cde188f38f6ac4677ae261da178f65e6ea5f72d/lib/ret_web/channels/hub_channel.ex#L395-L398
and you remove the room-host association if the room is older than a day and have no presence https://github.com/mozilla/reticulum/blob/0cde188f38f6ac4677ae261da178f65e6ea5f72d/lib/ret/hub.ex#L402-L417 and https://github.com/mozilla/reticulum/blob/22c2e4b3cd5fdbce1b5f8d8be6cf02abeca2b206/config/prod.exs#L126

[bemyevent]

Soory for this question , but if I have 2 or 3 room , do I need to launch 2 app ? I don't understand the app notion, sorry for this question

[Fi1osof]

@vincentfretin thanks for research!