Scalability & Performance

[holtwick]

Currently peer.school is implemented as a "full mesh network" i.e. each client is connected with each other. So even if only the teacher is streaming she has to send the same stream to as many peers as there are students online. The same for the student, if he is active.

There are multiple solutions to the issue:

1. Keep it that way and assume the teacher has enough bandwidth (unlikely)
2. Use a central server that distributes the video signal to the clients like https://jitsi.org/ (not end-to-end encrypted!)
3. Have each peer to proxy to a certain number of other peers (consumes bandwidth on student's side)

So all solutions have trade-offs. Here some links about the topic:

• Stackoverflow
• Jitsi not e2e
• Peers proxy demo implementation
• https://bloggeek.me/webrtc-rtcpeerconnection-one-per-stream/

[SebastianSpeitel]

Could we do something like a chain, where each node sends it's incoming stream to 2 or more (depending on available bandwith) other peers.

So each node has the required input streams but at max 2(configurable) outgoing streams

[holtwick]

Right, this is similar to this approach. But if one member of the chain fails or has low bandwidth it will create issues. I'm pretty sure somebody already solved that problem somewhere ;) @dmonad do you have any experiences with this kind of issue? At least for sending large files this problem seems to be solved: https://webtorrent.io/

[dmonad]

A central place for bundling all streams (like jitsi) is the most reliable approach. Creating a tree of streams is very unreliable in practice. A partially connected network of streams has more redundancy and the chance that data is not propagated is less likely. y-webrtc already supports partially connected networks. Although, at the moment, y-webrtc has no guarantees that the network will stay connected. But I showed at FOSDEM that this works with >100peers on an unreliable network connection.

Here are some ideas based on y-webrtc: You could only broadcast the stream of the user who is currently speaking through the network. In this case you could simply broadcast one stream, instead of multiplexing many streams. If you want to broadcast many streams, you'd need to multiplex all received streams with your current stream. This should be pretty simple even using y-webrtc (see stream manipulation using the canvas element / the only thing that will be hard to multiplex is audio data). A simple approach that solves multiplexing would be to share the streams using Y.Arrays and reduce the number of connections (because every peer broadcasts all updates, it would make sense to reduce the maximum number of peers to ~10 connections). The advantage would be that all streams are stored in the shared data type, so you could easily create a recording based on the real data or jump back to a previous point while listening. I think that Webtorrent would be too slow for a live conference - but it is definitely worth checking out.

[holtwick]

Thanks a lot for the detailed info @dmonad !

I think I will first try the solutions that give quick results. These are, as outlined by you:

1. Reduce the number of streams i.e. only stream if a student is talking to the class
2. Reduce the bandwdith by using small viewport sizes and few frames per second
3. Probably optionally/ by choice: Use Jitsi for video transmission. In future provide installations and installation instructions for on-premise solutions, to address privacy related issues.

For small groups it certainly allright to use WebRTC streaming directly. Optimizing for a sophisticated network topology and complex stream improvements is probably something to do, if this project becomes a huge success ;)

[holtwick]

I did some calculations based on a 1 teacher and 30 students basis per class and some real world tests:

• Each video stream is around 2 and 2,5 Mbit/s
• For each connected student the teacher needs to send the stream separately i.e. 2 * 30 = 60 Mbit/s

Assuming that the minimal setup at home could be a 16Mbit line, this is already exceeding the limits. If multiple teachers would stream from a school and lets say there are 10 classes streaming, this would be:

• 60 Mbit/s * 10 classes = 600 Mbit/s = 0,6 Gbit/s

We come to a limit around 16 classes per 1Gbit line.

Conclusion: peer.school should work fine for a limited number of participants, lets say 5 or maximal 8. For larger groups at least the video transmission needs to be fixed by a centralized service like Jitsi.