Speaker.app is a batteries-included, quasi-decentralized, alternative free speech audio platform that is compatible on any device that supports a modern web browser.
Rather than a centralized server providing proxying of streams from each participant to other participants (i.e. an MCU / SFU), one can choose to host a network (or "room") where others can connect to, either publicly or privately. The network hosting participant's web browser acts as the "server" for the other participants to connect to on the given network, and all proxying is done, including message storage and relaying, through that browser.
Public networks are visible in a "network discovery" view, which serves as the default homepage for the application.
No user accounts or passwords are required to join a public network, and user identities are generated using Ethereum, with a randomized user profile, by default. Users can change their user profile to their liking, while their profile information is stored locally via local storage.
To see it live, navigate to https://speaker.app.
- Speaker.app / zenRTC / Phantom Server
| Chrome | Edge (Chromium) | Firefox | Safari | |
|---|---|---|---|---|
| Android | ✓ | ✓ | ✓ | N/A |
| iOS | [transcoder host only] | N/A | N/A | ✓ |
| Linux | ✓ | ✓ | ✓ | N/A |
| macOS | ✓ | ✓ | ✓ | ✓ |
| Windows | ✓ | ✓ | ✓ | N/A |
Note, on every OS except iOS, Chrome is the recommended browser; On iOS, Safari should be used.
Frontend: Built with create-react-app; state is managed with multiple Providers and accessible via useContext hooks.
Backend: Node.js app, using Socket.io and Express. Cluster module is utilized to utilize multiple CPUs and a Redis store is utilize to scale Socket.io across the CPUs.
MongoDB: Network details (name, host, number of participants) are stored in MongoDB. When in development mode, Mongo Express is available at http://localhost:8081, and provides a web-based administrative interface.
Let's Encrypt: Free SSL certificates are managed via the linuxserver.io/docker-swag Docker image.
dev-ssl-proxy: In development, a self-signed SSL proxy is utilized in replacement of Let's Encrypt, to enable local development with SSL turned on (cam / mic / other HTML5-related APIs require SSL by default).
Coturn: A STUN / TURN server for WebRTC NAT traversal is included in the Docker Compose configuration, but is not enabled by default.
Included WebRTC Experiments: Within the source code are some previous real-time, shared experience experiments such as a drum looper, a sound sampler (play piano / electric guitar w/ keyboard), text-to-speech, TensorFlow-based skeletal tracker, and a game emulator.
These experiments are mostly dormant and commented-out, but have made for some interesting demos in the past and may be re-enabled in the future.
Mesh network example. (Illustration borrowed from simple-peer)
Most group-based WebRTC calls, which don't have a centralized MCU / SFU rely on each peer to send out an extra stream to multiple peer. This is not very efficient as for every participant added, every device connected must send out additional streams.
Centralized MCU / SFU example.
More advanced calling platforms utilize a centralized MCU / SFU. While this is more efficient in terms of the network, additional considerations, and money, are needed in order to scale out the backend infrastructure.
Using a topology similar to the MCU / SFU example above, Speaker.app attempts to solve the scalability issue without throwing a lot of extra money into hosting fees, by enabling individual participants to host their own networks, on their own hardware, using their own bandwidth, while at the same time providing greater privacy and flexibility.
zenRTC (built with simple-peer) is based on WebRTC, adding additional functionality such as user-level network strength indication, events over data channels, and P2P-based shared state syncing.
Phantom Server is a network host which runs in your web browser, and acts as the host, shared state manager, proxy, and transcoder for all connected participants within a WebRTC network.
Every participant connects to the Phantom Server via a P2P connection and Phantom Server handles the stream negotiations / network programming with the other peers.
Speaker.app is able to provide a quasi-decentralized MCU / SFU by enabling clients to run them in their own browsers, as a virtual machine.
*At the time of writing the Chrome on the Apple M1 processor is by far the most efficient for doing browser-based streaming transcoding, compared to a variety of Intel processors which have been tested on, though development has mostly been done on Intel processors / Linux. ARM is the future, it seems.
Network hosting has also been tested on non-optimal hardware (i.e. 2018 Samsung J2; Intel i3) with adequate results for streaming 4K video streams to 4 participants. Good hardware such as the new Apple M1 processor allows much greater yields, and better scalability.*
TLDR; Experimentation.
I was faced with a task for building a WebRTC bridge between two third party services in the virtual healthcare industry and after trying some various approaches, discovered that using a headless Chrome instance on the server was the path of least effort and less bugs to squash, though not necessarily being greatly efficient on its own.
Running a headless Chrome instance on the server is very versatile, in being that you've got a really solid WebRTC implementation baked in, with the ability to mix audio and video streams using JavaScript and the real DOM.
Wanting to continue pursuing the effort of a script-able WebRTC bridge using a web browser, and thinking of ways to potentially scale such a system, I made the decision to allow client-side devices to host these sessions, now no longer utilizing the headless Chrome instances as the main method of hosting sessions.
NOTE: If you wish to host your own network (or room) you DO NOT HAVE TO DO this, and can instead go https://speaker.app/setup/network/create and create your own network!
The following is ONLY if you wish to host the entire infrastructure yourself.
All environments require
- Bash (Unix shell) If running the included Bash build scripts
- Docker
- Docker Compose
Development environments require
- Node.js 12+
Some Bash scripts have been provided to help facilitate the starting and stopping of the respective environments. It is recommended to use these scripts instead of calling the Docker commands directly, as they will provide supplemental environment variables as well as any additional build instructions.
In development environments, most of the container volumes have a mount directly to the host so that the source code can be updated in the containers without rebuilding. See the respective docker.compose*.yml configurations and corresponding Dockerfile files for more details.
Set up the environment
Copy the sample environment.
$ cp .env.sample .envThen populate .env with the configuration relevant to your environment.
Note that other environment variables are set within the docker-compose*.yml files and are intended to be considered static.
To build the Docker containers
Note that development environments may require additional dependencies to be installed.
$ ./build.prod.sh # Or ./build.dev.sh, depending on environmentTo start the containers
$ ./start.prod.sh # Or ./start.dev.sh, depending on environmentTo stop the containers
This stops the containers and tears down their temporary storage.
$ ./stop.sh # Stops any environmentPublic networks can be discovered on the default home page. Private networks do not appear in the public network discovery but can be accessed via URL or QR code.
Testing can be performed by running:
$ ./test.shNote, development packages will be automatically installed locally when testing.
Jest tests / Manual tests via BrowserStack
Source-code contributions and forks are welcome. Open an issue if something needs to be addressed.
Related to scaling Socket.io across CPU cores. Make sure all npm installs are executed on the same platform as used during runtime. See lovell/farmhash#21
Solution: echo fs.inotify.max_user_watches=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p Reference: gatsbyjs/gatsby#11406 (note, Gatsby is not utilized in this project)
Leftover artifact when after doing some headless Chrome testing and seeing how the CPU was being throttled. Might be useful in the future.
lscpu | grep MHz
Update this document and have it automatically generate the table of contents in VS Code: Markdown All in One Extension
To contribute, however slightly, to the commonwealth of all human innovation and experience.
PayPal: https://www.paypal.com/paypalme/zenOSmosis
Buy Me a Coffee: https://www.buymeacoffee.com/Kg8VCULYI