Zenon reverse proxy

This is a reverse proxy websocket server targeting different functionalities:

1. Enable secure websocket functionality for a Zenon Node (see Setup)
2. Load balancing (see Configuration)
3. Client request caching

Functionalities 1 and 2 are already implemented; request caching implementation is one of the next steps.

Running this in front of a Zenon Node introduces some overhead, see Performance.

Setup

1. This proxy supports both ws and wss; if wss is required, a valid certificate must exist. An option is a free certificate from https://letsencrypt.org. To acquire one, install certbot:

   sudo snap install core; sudo snap refresh core
   sudo snap install --classic certbot
   sudo ln -s /snap/bin/certbot /usr/bin/certbot
   # open http port so that certbot can communicate; can be closed afterwards
   sudo ufw allow http
   # execute certbot and follow the instructions to retrieve an ssl certificate
   

   See also https://certbot.eff.org/instructions.

2. Install the systemd development files:

   sudo apt install libsystemd-dev
   

3. A number of thirdparty repositories are used. To initialize them, run git submodule update --init --recursive in the project root.

4. The websocket server is built on uWebSockets (https://github.com/uNetworking/uWebSockets). The code should now exist in thirdparty. If wss functionality shall be used, this library must be built with SSL-support once. From the projects root:

   cd thirdparty/uWebSockets/uSockets/ && make -j8 boringssl && cd ../../..
   cd thirdparty/uWebSockets/ && WITH_BORINGSSL=1 make -j8 && cd ../..
   

   If you don't want to use secure sockets, skip the first step and remove the WITH_BORINGSSL from the second.

5. Build this project.

   mkdir build && cd build && cmake -DCMAKE_BUILD_TYPE=Release ..
   sudo make -j8 install
   

   Replace Release with Debug if you e.g. want to run the test script (see Performance).

Installing

If you run with root privileges, you can install the tool: sudo make install. which znn-repro should then print the path it installed to, probably /usr/local/bin. This also installs a systemd service file.

Configuration

The resulting binary znn-repro has to be configured in a configuration file. You can use the config/create_configuration.py script to generate one. Note that you have to run that under the account that you will use to run znn-repro, i.e. if you want to use secure websockets, which require root privileges, first make sure to be logged in as root. Then run

python3 create_configuration.py

and input the required data. A config.json file will then be generated in /root/.config/znn-repro/config.json/. If you run it as regular user, the config file is stored in /home/<your-username>/.config/znn-repro. You can also edit this file manually.

The script will also attempt to open the required ports in the firewall, so if you edit it manually, don't forget to do that.

There is an example configuration in config:

{
    "proxies": [
        {
            "node": "ws://localhost:35998",
            "wss": false,
            "port": 8001,
            "timeout": 20
        },
        {
            "node": "ws://localhost:35998",
            "wss": false,
            "port": 8001,
            "timeout": 120
        }
    ],
    "certificates": ""
}

This configuration defines a server listening with two independent threads for Zenon Client connections. Both listen on port 8001 and both connect to the same node on localhost, listening for websocket connections on port 35998. Each proxy defines a timeout of 25 milliseconds for new connections to the Zenon Node.

A Zenon client that wanted to connect to your proxy would therefore have to connect to ws://<your-host>:8001.

Each proxy is not limited in the amount of websocket connections it accepts.

Defining several proxies is a way to scale your node, as these proxies operate independently. If they share the same node, connection to that node is the bottleneck

Running

./build/znn-repro. Remember that it expects the configuration file in your users' .config folder.

If you want to run it as a background service (see #installing), start the system daemon: systemd start znn-repro. The configuration must then exist in /root/.config/. Check the output of systemd status znn-repro. If your go-zenon service was running already, znn-repro should be active as well now.

Tests

A python script lives in directory test. Put that on a client computer and use it to test response times for different scenarios:

1. Multiple clients connect concurrently, each sending a single request.
2. A single client connects and sends multiple requests concurrently.
3. Multiple clients connect from different processes, each sending multiple requests concurrently.

See the help text the script prints when started without argument -h.

Performance

The following lists some performance results taken by the python test script, started by the following command:

python3 py-ws.py ws://redacted.domain:<port> -m pcc -n <amount> -r <requests>

where <port> is either 8001 (ws), 443 (wss) or 39558 (direct). <amount> is the number of parallel clients (processes), <requests> is the number of requests and responses each sends and receives, respectively.

Every test was run with all proxies listening on the same port, each connecting to the node on ws://localhost:35998. timeout (node connection timeout in ms) and wss are the corresponding values from the configuration. Errors is the amount of clients that failed with TimeoutError during connection. Duration is the duration in seconds the whole script execution took, i.e. the time it took <amount> clients to send/receive 1000 requests/responses.

Proxies	timeout	wss	clients	requests	Duration
1	20	no	10	1000	37
1	100	no	100	1000	379
1	100	yes	100	1000	336
10	50	no	10	1000	30
10	50	no	100	1000	83
10	50	yes	10	1000	30
10	75	yes	100	1000	75
20	50	no	10	1000	30
20	75	no	100	1000	67
20	75	yes	100	1000	63
100	75	no	10	1000	30
100	120	no	100	1000	60
50	100	yes	100	1000	73
50	100	no	100	10000	605

Note that the timeout value has to be increased with the amount of proxies; you will receive the random timeout from node connections else. This is, however, only relevant if a lot of clients connect simultaneously.

Interestingly, the wss configuration is sometimes faster than the ws equivalent.

To contrast these results with a direct connection, the same script was run, connecting directly to the node (ws only):

• 10 clients, 1000 requests each: 29 seconds
• 100 clients, 1000 requests each: 49 seconds
• 100 clients, 10000 requests each: 465 seconds