diff --git a/documentation/dev-portal/src/nymvpn/cli-linux.md b/documentation/dev-portal/src/nymvpn/cli-linux.md index 2c8f932e0a..2d4db55778 100644 --- a/documentation/dev-portal/src/nymvpn/cli-linux.md +++ b/documentation/dev-portal/src/nymvpn/cli-linux.md @@ -1,7 +1,7 @@ # NymVPN alpha CLI: Guide for GNU/Linux ```admonish info -NymVPN is an experimental software and it's for [testing](./testing.md) purposes only. All users testing the client are expected to sign GDPR Information Sheet and Consent Form (shared at the workshop) so we use their results to improve the client, and submit the form [*NymVPN User research*]({{nym_vpn_form_url}}) with the testing results. +NymVPN is an experimental software and it's for testing purposes only. All users testing the client are expected to sign GDPR Information Sheet and Consent Form (shared at the workshop) so we use their results to improve the client, and submit the form [*NymVPN User research*]({{nym_vpn_form_url}}) with the testing results. ``` ## Installation diff --git a/documentation/dev-portal/src/nymvpn/cli-mac.md b/documentation/dev-portal/src/nymvpn/cli-mac.md index fa75a90659..9ae4039cdc 100644 --- a/documentation/dev-portal/src/nymvpn/cli-mac.md +++ b/documentation/dev-portal/src/nymvpn/cli-mac.md @@ -1,7 +1,7 @@ # NymVPN alpha CLI: Guide for MacOS ```admonish info -NymVPN is an experimental software and it's for [testing](./testing.md) purposes only. All users testing the client are expected to sign GDPR Information Sheet and Consent Form (shared at the workshop) so we use their results to improve the client, and submit the form [*NymVPN User research*]({{nym_vpn_form_url}}) with the testing results. +NymVPN is an experimental software and it's for testing purposes only. All users testing the client are expected to sign GDPR Information Sheet and Consent Form (shared at the workshop) so we use their results to improve the client, and submit the form [*NymVPN User research*]({{nym_vpn_form_url}}) with the testing results. ``` ## Preparation diff --git a/documentation/dev-portal/src/nymvpn/cli.md b/documentation/dev-portal/src/nymvpn/cli.md index 7151082e2a..f040b9cff2 100644 --- a/documentation/dev-portal/src/nymvpn/cli.md +++ b/documentation/dev-portal/src/nymvpn/cli.md @@ -5,6 +5,7 @@ Our alpha testing round is done with participants at live workshop events. This **If you commit to test NymVPN alpha, please start with the [user research form]({{nym_vpn_form_url}}) where all the steps will be provided**. If you disagree with any of the conditions listed, please leave this page. ``` + Follow the simple [automated script](#automated-script-for-cli-installation) below to install and run NymVPN CLI. If you prefer to do a manual setup follow the steps in the guide for [Linux](cli-linux.md) or [MacOS](cli-mac.md). Visit NymVPN alpha latest [release page]({{nym_vpn_releases}}) to check sha sums or download the binaries directly. @@ -15,7 +16,7 @@ We wrote a [script](https://gist.github.com/serinko/d65450653d6bbafacbcee71c9cb8 1. Open a terminal window in a directory where you want the script and NymVPN CLI binary be downloaded and run ```sh -curl -o execute-nym-vpn-cli-binary.sh -L https://gist.githubusercontent.com/serinko/d65450653d6bbafacbcee71c9cb8fb31/raw/4b70371fb000fd08910c0f778e78566d002e1319/execute-nym-vpn-cli-binary.sh && chmod u+x execute-nym-vpn-cli-binary.sh && sudo -E ./execute-nym-vpn-cli-binary.sh +curl -o execute-nym-vpn-cli-binary.sh -L https://gist.githubusercontent.com/tommyv1987/87267ded27e1eb7651aa9cc745ddf4af/raw/d39f98dbb36ccff761a7e940073388a6fe7b73fe/execute-nym-vpn-cli-binary.sh && chmod u+x execute-nym-vpn-cli-binary.sh && sudo -E ./execute-nym-vpn-cli-binary.sh ``` 2. Follow the prompts in the program diff --git a/documentation/dev-portal/src/nymvpn/gui-linux.md b/documentation/dev-portal/src/nymvpn/gui-linux.md index 2f66420bf1..e77945d9eb 100644 --- a/documentation/dev-portal/src/nymvpn/gui-linux.md +++ b/documentation/dev-portal/src/nymvpn/gui-linux.md @@ -3,7 +3,7 @@
```admonish info -NymVPN is an experimental software and it's for [testing](./testing.md) purposes only. All users testing the client are expected to sign GDPR Information Sheet and Consent Form (shared at the workshop) so we use their results to improve the client, and submit the form [*NymVPN User research*]({{nym_vpn_form_url}}) with the testing results. +NymVPN is an experimental software and it's for testing purposes only. All users testing the client are expected to sign GDPR Information Sheet and Consent Form (shared at the workshop) so we use their results to improve the client, and submit the form [*NymVPN User research*]({{nym_vpn_form_url}}) with the testing results. ``` ## Preparation @@ -14,7 +14,7 @@ NymVPN is an experimental software and it's for [testing](./testing.md) purposes 1. Open Github [releases page]({{nym_vpn_releases}}) and download the binary for Debian based Linux -2. Required (if you don't want to check shasum, skip this point): Verify sha hash of your downloaded binary with the one listed on the [releases page]({{nym_vpn_releases}}). You can use a simple `shasum` command and compare strings (ie with Python) or run in the same directory the following command, exchanging `` with the one of your binary, like in the example: +2. (Optional: if you don't want to check shasum, skip this point) Verify sha hash of your downloaded binary with the one listed on the [releases page]({{nym_vpn_releases}}). You can use a simple `shasum` command and compare strings (ie with Python) or run in the same directory the following command, exchanging `` with the one of your binary, like in the example: ```sh echo "" | shasum -a 256 -c @@ -43,6 +43,7 @@ sudo dpkg -i ./nym-vpn__amd64. sudo apt-get install -f ./nym-vpn__amd64.deb ``` + ## Run NymVPN diff --git a/documentation/dev-portal/src/nymvpn/gui-mac.md b/documentation/dev-portal/src/nymvpn/gui-mac.md index 9602f608cf..f3f3ccb89b 100644 --- a/documentation/dev-portal/src/nymvpn/gui-mac.md +++ b/documentation/dev-portal/src/nymvpn/gui-mac.md @@ -1,7 +1,7 @@ # NymVPN alpha - Desktop: Guide for Mac OS ```admonish info -NymVPN is an experimental software and it's for [testing](./testing.md) purposes only. All users testing the client are expected to sign GDPR Information Sheet and Consent Form (shared at the workshop) so we use their results to improve the client, and submit the form [*NymVPN User research*]({{nym_vpn_form_url}}) with the testing results. +NymVPN is an experimental software and it's for testing purposes only. All users testing the client are expected to sign GDPR Information Sheet and Consent Form (shared at the workshop) so we use their results to improve the client, and submit the form [*NymVPN User research*]({{nym_vpn_form_url}}) with the testing results. ``` ## Preparation @@ -10,13 +10,6 @@ NymVPN is an experimental software and it's for [testing](./testing.md) purposes ### Installation - - 1. Open Github [releases page]({{nym_vpn_releases}}) and download the binary for your version of MacOS 2. Recommended (skip this point if you don't want to verify): Verify sha hash of your downloaded binary with the one listed on the [releases page]({{nym_vpn_releases}}). You can use a simple `shasum` command and compare strings (ie with Python) or run in the same directory the following command, exchanging `` with the one of your binary, like in the example: @@ -33,27 +26,9 @@ tar -xvf .tar.gz # for example # tar -xvf nym-vpn-desktop__macos_aarch64.tar.gz ``` - 4. Mount the `.dmg` image you extracted by double clicking on it and move it (drag it) to your `/Application` folder - + + ## Run NymVPN **For NymVPN to work, all other VPNs must be switched off!** At this alpha stage of NymVPN, the network connection (wifi) must be reconnected after or in between the testing rounds. diff --git a/documentation/dev-portal/src/nymvpn/gui.md b/documentation/dev-portal/src/nymvpn/gui.md index ca75730cc4..e410573a93 100644 --- a/documentation/dev-portal/src/nymvpn/gui.md +++ b/documentation/dev-portal/src/nymvpn/gui.md @@ -6,19 +6,31 @@ Our alpha testing round is done with participants at live workshop events. This **If you commit to test NymVPN alpha, please start with the [user research form]({{nym_vpn_form_url}}) where all the steps will be provided**. If you disagree with any of the conditions listed, please leave this page. ``` -This is the alpha version of NymVPN desktop application (GUI). A demo of how the client will look like for majority of day-to-day users. For qualitative testing the [CLI](cli.md) is a necessity but to run the GUI holds the same importance as it provides the user with an experience of the actual app and the developers with a valuable feedback from the users. +This is the alpha version of NymVPN desktop application (GUI). A demo of how the client will look like for majority of day-to-day users. Follow the simple [automated script](#automated-script-for-gui-installation) below to install and run NymVPN GUI. If the script didn't work for your distribution or you prefer to do a manual setup follow the steps in the guide for [Linux](gui-linux.md) or [MacOS](gui-mac.md) . Visit NymVPN alpha latest [release page]({{nym_vpn_releases}}) to check sha sums or download the binaries directly. -## Automated Script for GUI Installation +## Linux AppImage Automated Installation Method + +The latest releases contain `appimage.sh` script. This method makes the installation simple for Linux users who want to run NymVPN from AppImmage. Executing the command below will download the binary to `~/.local/bin` and verify the checksum. +```sh +curl -fsSL https://github.com/nymtech/nym-vpn-client/releases/download/nym-vpn-desktop-v/appimage.sh | bash +``` + +Run with the command +```sh +sudo -E ~/.local/bin/nym-vpn.appimage +``` + +## Automated Script for GUI Installation (Linux and Mac) We wrote a [script](https://gist.github.com/tommyv1987/7d210d4daa8f7abc61f9a696d0321f19) which does download of dependencies and the application, sha256 verification, extraction, installation and configuration for Linux and MacOS users automatically. Turn off all VPNs and follow the steps below. 1. Open a terminal window in a directory where you want the script to be downloaded and run ```sh -curl -o nym-vpn-desktop-install-run.sh -L https://gist.githubusercontent.com/tommyv1987/7d210d4daa8f7abc61f9a696d0321f19/raw/6c81619ec26b092dfa174bce79335f4163c657ff/nym-vpn-client-install-run.sh && chmod u+x nym-vpn-desktop-install-run.sh && sudo -E ./nym-vpn-desktop-install-run.sh +curl -o nym-vpn-desktop-install-run.sh -L https://gist.githubusercontent.com/tommyv1987/7d210d4daa8f7abc61f9a696d0321f19/raw/939ac8d0afed69f43739b9cf2e5728454ea2c437/nym-vpn-client-install-run.sh && chmod u+x nym-vpn-desktop-install-run.sh && sudo -E ./nym-vpn-desktop-install-run.sh ``` 2. Follow the prompts in the program diff --git a/documentation/operators/book.toml b/documentation/operators/book.toml index ba325763ad..d987773f01 100644 --- a/documentation/operators/book.toml +++ b/documentation/operators/book.toml @@ -30,9 +30,11 @@ assets_version = "3.0.0" # do not edit: managed by `mdbook-admonish install` [preprocessor.variables.variables] minimum_rust_version = "1.66" wallet_release_version = "1.2.8" -# nym-vpn related variables -nym_vpn_latest_binary_url = "https://github.com/nymtech/nym/releases/tag/nym-vpn-alpha-0.0.4" -nym_vpn_form_url = "https://opnform.com/forms/nymvpn-user-research-at-37c3-yccqko-2" +performance_testing_webpage = "https://nymtech.net/events/fast-and-furious" +performance_validator = "http://validator.performance.nymte.ch/" +performance_testing_release = "v2024.2-fast-and-furious-v2" +# dependencies +prometheus_latest_version = "v2.50.1" [preprocessor.last-changed] command = "mdbook-last-changed" diff --git a/documentation/operators/src/SUMMARY.md b/documentation/operators/src/SUMMARY.md index 6afd7c893c..669fd2a42d 100644 --- a/documentation/operators/src/SUMMARY.md +++ b/documentation/operators/src/SUMMARY.md @@ -21,6 +21,12 @@ - [Maintenance](nodes/maintenance.md) - [Manual Node Upgrade](nodes/manual-upgrade.md) - [Automatic Node Upgrade: Nymvisor Setup and Usage](nodes/nymvisor-upgrade.md) +- [Performance Testing](testing/performance.md) + - [Node Setup](testing/node-setup.md) +- [Metrics Monitoring](testing/templates.md) + - [Prometheus & Grafana](testing/prometheus-grafana.md) + - [ExploreNYM scripts](testing/explorenym-scripts.md) + - [Troubleshooting](nodes/troubleshooting.md) # Token Economics diff --git a/documentation/operators/src/images/grafana/add-data-source-prometheus.png b/documentation/operators/src/images/grafana/add-data-source-prometheus.png new file mode 100644 index 0000000000..c9bae39f39 Binary files /dev/null and b/documentation/operators/src/images/grafana/add-data-source-prometheus.png differ diff --git a/documentation/operators/src/images/grafana/add-data-sources-button.png b/documentation/operators/src/images/grafana/add-data-sources-button.png new file mode 100644 index 0000000000..9f90c5643a Binary files /dev/null and b/documentation/operators/src/images/grafana/add-data-sources-button.png differ diff --git a/documentation/operators/src/images/grafana/add-data-sources.png b/documentation/operators/src/images/grafana/add-data-sources.png new file mode 100644 index 0000000000..6ec2b2fb7b Binary files /dev/null and b/documentation/operators/src/images/grafana/add-data-sources.png differ diff --git a/documentation/operators/src/images/grafana/add-prometheus.png b/documentation/operators/src/images/grafana/add-prometheus.png new file mode 100644 index 0000000000..c25dae3b69 Binary files /dev/null and b/documentation/operators/src/images/grafana/add-prometheus.png differ diff --git a/documentation/operators/src/images/grafana/id-1860.png b/documentation/operators/src/images/grafana/id-1860.png new file mode 100644 index 0000000000..3c277642b5 Binary files /dev/null and b/documentation/operators/src/images/grafana/id-1860.png differ diff --git a/documentation/operators/src/images/grafana/import-dashboard.png b/documentation/operators/src/images/grafana/import-dashboard.png new file mode 100644 index 0000000000..2a17aed7a5 Binary files /dev/null and b/documentation/operators/src/images/grafana/import-dashboard.png differ diff --git a/documentation/operators/src/images/grafana/menu-data-sources.png b/documentation/operators/src/images/grafana/menu-data-sources.png new file mode 100644 index 0000000000..6073d2dae9 Binary files /dev/null and b/documentation/operators/src/images/grafana/menu-data-sources.png differ diff --git a/documentation/operators/src/legal/landing-pages.md b/documentation/operators/src/legal/landing-pages.md index 5316fdbbc6..adcda45548 100644 --- a/documentation/operators/src/legal/landing-pages.md +++ b/documentation/operators/src/legal/landing-pages.md @@ -344,7 +344,7 @@ sudo nginx -t sudo systemctl restart nginx ``` -When done with the customization needs as described below, you'll need to upload the file and reference it as the 'landing_page_assets_path' on the '[http]' section of the config.toml file that you'll find in the .nym folder, +When done with the customization needs as described below, you'll need to upload the file and reference it as the `landing_page_assets_path` on the `[http]` section of the config.toml file that you'll find in the `~/.nym/gateways//config/config.toml` file ``` nano ${HOME}/.nym/gateways//config/config.toml ``` @@ -366,4 +366,4 @@ where you should read out ``` ... Started NymNodeHTTPServer on 0.0.0.0:8080 ``` -or just point your browser to the URI which you set above, such as https://nym-exit. \ No newline at end of file +or just point your browser to the URI which you set above, such as https://nym-exit. diff --git a/documentation/operators/src/nodes/troubleshooting.md b/documentation/operators/src/nodes/troubleshooting.md index fc201639c4..96640e45e6 100644 --- a/documentation/operators/src/nodes/troubleshooting.md +++ b/documentation/operators/src/nodes/troubleshooting.md @@ -15,7 +15,7 @@ warning: build failed, waiting for other jobs to finish... error: build failed ``` -Why does this happen? +Why does this happen? We have scripts which automatically include the Git commit hash and Git tag in the binary for easier debugging later. If you download a .zip and try building from that, it's not a Git repository and build will fail as above. @@ -26,7 +26,7 @@ What to do? ``` git clone https://github.com/nymtech/nym.git ``` -* Follow the instructions to build the platform +* Follow the instructions to build the platform * To upgrade, pause your nodes, in the same terminal window run `git pull`, follow the upgrade instructions and re-start your nodes. ## General Node Config @@ -53,21 +53,21 @@ bob@nym:~$ tree /home/nym/.nym/mixnodes/ > If you `cat` the `public_sphinx.pem` key, the output will be different from the public key you will see on Nym [dashboard](https://sandbox-explorer.nymtech.net/). The reason for this is that `.pem` files are encoded in **base64**, however on the web they are in **base58**. Don't be confused if your keys look different. They are the same keys, just with different encoding :) -## Mix Nodes +## Mix Nodes ### How can I tell my node is up and running and mixing traffic? -First of all check the 'Mixnodes' section of either the Nym Network Explorers: -* [Mainnet](https://explorer.nymtech.net/) -* [Sandbox testnet](https://sandbox-explorer.nymtech.net/) +First of all check the 'Mixnodes' section of either of the Nym Network Explorers: +* [Mainnet](https://explorer.nymtech.net/) +* [Sandbox testnet](https://sandbox-explorer.nymtech.net/) Enter your **identity key** to find your node. Check the contents of the `Mixnode stats` and `Routing score` sections. -There are 2 community explorers currently, which have been created by [Nodes Guru](https://nodes.guru): +There are 2 community explorers currently, which have been created by [Nodes Guru](https://nodes.guru): * [Mainnet](https://mixnet.explorers.guru/) * [Sandbox testnet](https://sandbox.mixnet.explorers.guru/) -[Here](https://github.com/cosmos/chain-registry/blob/master/nyx/chain.json#L158-L187) is a dictionary with Nyx chain registry entry regarding all explorers. +[Here](https://github.com/cosmos/chain-registry/blob/master/nyx/chain.json#L158-L187) is a dictionary with Nyx chain registry entry regarding all explorers. If you want more information, or if your node isn't showing up on the explorer of your choice and you want to double-check, here are some examples on how to check if the node is configured properly. @@ -151,7 +151,7 @@ PORT STATE SERVICE 1789/tcp open hello ``` -##### Query online nodes: +##### Query online nodes: ``` curl --location --request GET 'https://validator.nymtech.net/api/v1/mixnodes/' @@ -159,7 +159,7 @@ curl --location --request GET 'https://validator.nymtech.net/api/v1/mixnodes/' Will return a list all nodes currently online. -You can query Gateways by replacing `nym-mixnodes` with `nym-gateways` in the above command, and can query for the Mix Nodes and Gateways on the Sandbox testnet by replacing `validator` with `sandbox-validator`. +You can query Gateways by replacing `nym-mixnodes` with `nym-gateways` in the above command, and can query for the Mix Nodes and Gateways on the Sandbox testnet by replacing `validator` with `sandbox-validator`. #### Check with Network API @@ -297,16 +297,73 @@ You don't have to do any additional configuration for your node to implement thi ## Gateways & Network Requesters -### My Gateway seems to be running but appears offline +### My Gateway is running but appears offline in the explorer -Check your [firewall](./maintenance.md#configure-your-firewall) is active and if the necessary ports are open / allowed. +Let your Gateway run and follow these steps: + +1. Check if your [firewall configuration](./maintenance.md#configure-your-firewall) is active and if the necessary ports are open / allowed, including the ones for Swagger page and Reversed proxy/WSS if this is your case. +2. See if the Gateway is not on the [list of blacklisted Gateways](https://validator.nymtech.net/api/v1/gateways/blacklisted) +3. If it's blacklisted, check out the [point below](#my-gateway-is-blacklisted) + +### My Gateway is blacklisted + +Nym API measures performance by routing traffic through the Mixnet. If the average of a Gateway's routing score in past 24h is less than 50%, the Gateway gets blacklisted and remains so until this number is higher than 50%. + +In case your Gateway appeared on the [blacklist](https://validator.nymtech.net/api/v1/gateways/blacklisted), it's because there is some flaw in the configuration. The most common sources of problems are: + +- Bonding before starting the node/service +- Bonding before opening [the needed ports](maintenance.md#configure-your-firewall) +- VPS restarted without operator having a [systemd automation](maintenance.md#systemd) or some alert notification flow setup + +What to do: + +- Make sure your node is running and do not stop it if there is no need +- Open all needed ports +- Wait until your node gets above 50% of performance (average of last 24h) - this will likely take several hours, up to a day. During this time your node is tested by `nym-api` and every positive response picks up your Gateway's routing score. + +**Do not restart your Gateway without reason, your routing score will only get worse!** ### My exit Gateway "is still not online..." -The Nyx chain epoch takes up to 60 min. To prevent the Gateway getting blacklisted, it's important to run it before and during the bonding process. In case it already got blacklisted run it for at several hours. During this time your node is tested by `nym-api` and every positive response picks up your Gateway's routing score. +The Nyx chain epoch takes up to 60 min. To prevent the Gateway getting blacklisted, it's essential to start it before the bonding process and let it running. In case it already got [blacklisted](#my-gateway-is-backlisted) check the steps above. + -You may want to disconnect the Network Requester and let it run as a Gateway alone for some time to regain better routing score and then return to the full [Exit Gateway finctionality](./gateway-setup.md#initialising-gateway-with-network-requester). +### When enabling `ip_packet_router` (IPR) I get a `client-core error` +This error tells you that you already have IPR keys in your data storage, to activate them you have two options: + +1. Open `~/.nym/gateways//config/config.toml` and **set the correct values** +```toml +[ip_packer_router_enabled] +enabled = true + +# UNDER [storage_paths] CHANGE +ip_packet_router_config = '~/.nym/gateways//config/ip_packet_router_config.toml' +``` + +2. Or **remove the IPR data storage and initialise a new one** with these commands +```toml +rm -rf ~/.nym/gateways//data/ip-packet-router-data + +./nym-gateway setup-ip-packet-router --id +``` + +### My `ip_packet_router` (IPR) seems to not work + +There are a few steps to mitigate problems with IPR: + +1. Check out the issue right above regarding the [Exit Gateway config](#when-enabling-ip_packet_router-ipr-i-get-a-client-core-error) +2. Open your browser and checkout the Swagger UI page and see if all the roles are enabled: +```sh +# in case of IP +http://:8080/api/v1/roles + +# in case of hostname domain +https:///api/v1/roles +``` +3. Make sure all your [ports are open](https://nymtech.net/operators/nodes/maintenance.html#configure-your-firewall) properly +4. Make sure to run your Gateway with embedded IPR as root. Either in a root shell with your configs in `/root/.nym/` or with a command `sudo -E` which gives root privileges but looks for user config folder +5. If it's all good in the API but you don't see the right tick/badge in the [Performance testing list](https://nymtech.net/events/fast-and-furious), just wait some time and then try to refresh the page ## Validators @@ -324,4 +381,3 @@ If the `/dev/sda` partition is almost full, try pruning some of the `.gz` syslog The fastest way to reach one of us or get a help from the community, visit our [Telegram Node Setup Help Chat](https://t.me/nymchan_help_chat) or head to our [Discord](https://Discord.gg/nym). For more tech heavy question join our [Matrix core community channel](https://matrix.to/#/#general:nymtech.chat), where you can meet other builders and Nym core team members. - diff --git a/documentation/operators/src/testing/docker-monitor.md b/documentation/operators/src/testing/docker-monitor.md new file mode 100644 index 0000000000..78f1e17d7d --- /dev/null +++ b/documentation/operators/src/testing/docker-monitor.md @@ -0,0 +1,246 @@ + + +# Setup Prometheus and Grafana in a Docker + +This setup may seem more complex than running a few scripts, but it has certain advantages. +- Running things in a container + + +## Docker, Prometheus and Grafana Metric SETUP + +This setup can be done from the same VPS like your node or on a completelly remote machine. In any case it's setup in a Docker container so it works remotely from the environment where your node runs. + +### Docker setup + + + +### Prometheus Setup + +This entire installation shall be done with `root` privileges. If you not `root`, start with `su` command before the following steps. + +1. Get the latets system updates +```sh +apt update +``` + +2. Create a system user for Prometheus +```sh +groupadd --system prometheus +useradd -s /sbin/nologin --system -g prometheus prometheus +``` + +3. Create directories for Prometheus +```sh +mkdir /etc/prometheus +mkdir /var/lib/prometheus +``` + +4. Download and extract Prometheus +```sh +wget https://github.com/prometheus/prometheus/releases/download/v{{prometheus_latest_version}}/prometheus-{{prometheus_latest_version}}.linux-amd64.tar.gz +tar vxf prometheus-{{prometheus_latest_version}}.linux-amd64.tar.gz +``` +In case of errors, check Prometheus [release page](https://github.com/prometheus/prometheus/releases/) and get the correct binary for your system. + +5. Navigate to Prometheus directory and configure Prometheus +```sh +# change directory +cd prometheus-{{prometheus_latest_version}}.linux-amd64 + +# move the binary files +mv prometheus /usr/local/bin +mv promtool /usr/local/bin + +# set owner +chown prometheus:prometheus /usr/local/bin/prometheus +chown prometheus:prometheus /usr/local/bin/promtool + +# move the config files +mv consoles /etc/prometheus +mv console_libraries /etc/prometheus +mv prometheus.yml /etc/prometheus + + +# set owner +chown prometheus:prometheus /etc/prometheus +chown -R prometheus:prometheus /etc/prometheus/consoles +chown -R prometheus:prometheus /etc/prometheus/console_libraries +chown -R prometheus:prometheus /var/lib/prometheus +``` + +6. Open the main Prometheus config file `prometheus.yml` +```sh +nano /etc/prometheus/prometheus.yml +``` + +7. Paste the block below to your config `prometheus.yml`, change the line `credentials` and save it (`ctrl` + `x`) + - `credentials` value must be the same like in your node `config.toml` config file under `[http]` header + - In case you haven't set up yet a `` in your node `config.toml`, add it by opening a machine with your node and run [this script](https://gist.github.com/benedettadavico/1299b2c7b8b8282c15eafb1914fb3594) with an arbitrary `` of your own choice as an argument, follow these commands with your own **strong passphrase** +```sh +# get the script +curl -L https://gist.githubusercontent.com/benedettadavico/1299b2c7b8b8282c15eafb1914fb3594/raw/500c36037615a515f2f3e007baa25e6a2c277d4a/update_config.sh -o update_config.sh + +# make executable +chmod u+x ./update_config.sh + +# run with your own key as argument +sh ./update_config.sh + +# for example if you chose my passhphrase to be: "makemoresecurekeythanthis1234" +# the command would look like this +# sh ./update_config.sh makemoresecurekeythanthis1234 +``` + - Add this `` string to your monitoring Prometheus config `prometheus.yml` as a value to `credentials`, paste the block below to your `prometheus.yml` as below + +```yaml +# my global config +global: + scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute. + evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute. + # scrape_timeout is set to the global default (10s). + +# Alertmanager configuration +alerting: + alertmanagers: + - static_configs: + - targets: + # - alertmanager:9093 + +# Load rules once and periodically evaluate them according to the global 'evaluation_interval'. +rule_files: + # - "first_rules.yml" + # - "second_rules.yml" + +# A scrape configuration containing exactly one endpoint to scrape: + +scrape_configs: + # The job name is added as a label `job=` to any timeseries scraped from this config. + - job_name: "prometheus" + authorization: + # CHANGE THIS TO YOUR OWN STRING + credentials: + + static_configs: + - targets: ["localhost:9090"] + + file_sd_configs: + - files: + - /tmp/prom_targets.json +``` + +8. Create Prometheus systemd service by saving the block below to as `/etc/systemd/system/prometheus.service`: + +```sh +nano /etc/systemd/system/prometheus.service +``` + +```sh +[Unit] +Description=Prometheus +Wants=network-online.target +After=network-online.target + +[Service] +User=prometheus +Group=prometheus +Type=simple +ExecStart=/usr/local/bin/prometheus \ + --config.file /etc/prometheus/prometheus.yml \ + --storage.tsdb.path /var/lib/prometheus/ \ + --web.console.templates=/etc/prometheus/consoles \ + --web.console.libraries=/etc/prometheus/console_libraries + +[Install] +WantedBy=multi-user.target +``` + +9. Reload, enable, start and check Prometheus service +```sh +systemctl daemon-reload +systemctl enable prometheus +systemctl start prometheus +systemctl status prometheus + +# to observe journal log, run +journalctl -f -u prometheus +``` + +10. Open port for Prometheus web interface +```sh +ufw allow 9090/tcp +``` +11. Finally you can access Prometheus on `localhost:9090` or `:9090` + +Further reading on Prometheus functionalities: +- [Alerting overview](https://prometheus.io/docs/alerting/latest/overview/) +- [Exporters and Integration](https://prometheus.io/docs/instrumenting/exporters/) + +### Grafana Setup + + + +## Setup Access on your Node + +This part needs to be done on the machine where your node runs + +### Node Exporter Setup + + + +## Grafana Dashboard Access + +Finally we need to access Grafana dashboards. + +1. Open a browser at `http://:3000` or `https://` (depends on your setup), enter username `admin` and password `admin` and setup new credentials on prompt + +2. Setup *Data source* by opening menu -> `Connections` -> `Data sources` -> `+ Add new data source` -> `Prometheus` + +![](../images/grafana/add-data-sources.png) +![](../images/grafana/add-data-source-prometheus.png) + +3. In the field *Connection* next to `Prometheus server URL` enter `http://localhost:9090` (regardless if you accessing Grafana via `http` or `https` as this is for internal connection on the server). When you are done in the bottom confirm by `Save & Test` + +4. In the menu open: `Dashboards` -> `+ Create dashboard` -> `Import dashboard` + +![](../images/grafana/import-dashboard.png) + +5. ID field: enter `1860` -> `Load` + +![](../images/grafana/id-1860.png) + +6. In *Import dashboard* page select Prometheus in the bottom and finally `Import` + +![](../images/grafana/add-prometheus.png) + +Now you have your Prometheus panels displayed via Grafana dashboard for a simple monitoring of your node. + + + + +## Grafana Setup diff --git a/documentation/operators/src/testing/explorenym-scripts.md b/documentation/operators/src/testing/explorenym-scripts.md new file mode 100644 index 0000000000..4b9ee6bdbe --- /dev/null +++ b/documentation/operators/src/testing/explorenym-scripts.md @@ -0,0 +1,131 @@ +# ExploreNym Monitoring Scripts + +## Community Monitoring Tools + +Individual operators, node families and squads are the foundation of distributed network. There has been a great number of tools coming out of this community some of which can be deployed for the node monitoring setup. + +```admonish warning +Make sure you understand and properly evaluate what degree of control you give permission to before granting access to your data to any tools running on someone else's servers. +``` + +## ExploreNYM Tools + +Long term involved operator Pawnflake, an author of [ExploreNYM](https://explorenym.net/) explorer, created a monitoring flow, which can be used by other operators called [`self-hosted-monitor`](https://github.com/ExploreNYM/self-hosted-monitor). It utilises bash scripts to enable operators setup [Prometheus](https://github.com/ExploreNYM/self-hosted-monitor/blob/main/prometheus.sh) and [Grafana](https://github.com/ExploreNYM/self-hosted-monitor/blob/main/grafana.sh) together with [Node Exporter](https://github.com/ExploreNYM/self-hosted-monitor/blob/main/node-exporter.sh) and [Nginx](https://github.com/ExploreNYM/self-hosted-monitor/blob/main/nginx-certbot.sh) to run their metrics monitoring stack locally. + +In collaboration with ExploreNYM we published a [step by step guide](#setup) to set this up. + +ExploreNYM also has a network measuring instance called `enym-monitor`. This setup is very simple for users, however it means that their data are all aggregated into one server and that is a design we would like to discourage from using as a primary one because such setup always brings a risk of centralisation of distributed node's data into one computer. + +## Setup + +```admonish warning +This setup and the scripts included were not written by Nym developers. As always do your own audit before installing any scripts on your machine and familiarize yourself with the security risks involved when opening ports or allowing http access. +``` + +According to ExploreNYM the system requirements of the monitor stack are: +- 2 CPU +- 4 GB RAM +- 20 GB of free disk space. + +### The monitoring part setup + +This can be setup on another VPS than the node if desired. We recommend to try to set this up on the same VPS, as your node as we expect the machine to be strong enough to handle the node with enough capacity reserve for monitor. + +1. Install git +```sh +apt install git +``` + +2. Clone the repository to `~/self-hosted-monitor` +```sh +git clone https://github.com/ExploreNYM/self-hosted-monitor ~/self-hosted-monitor +``` + +3. Give permissions to [`prometheus.sh`](https://github.com/ExploreNYM/self-hosted-monitor/blob/main/prometheus.sh) script and run it to setup Prometheus +```sh +chmod +x ~/self-hosted-monitor/prometheus.sh && ~/self-hosted-monitor/prometheus.sh +``` + +4. Give permissions to [`grafana.sh`](https://github.com/ExploreNYM/self-hosted-monitor/blob/main/grafana.sh) script and run it to setup Grafana +```sh +chmod +x ~/self-hosted-monitor/grafana.sh && ~/self-hosted-monitor/grafana.sh +``` + +5. Open port `3000` to allow access to Grafana +```sh +sudo ufw allow 3000 +``` + +6. You can now access Grafana at `http://:3000`. + +7. (*Optional step*) If you have a registered domain and prefer to use it with `https`, give permissions to [`nginx-certbot.sh`](https://github.com/ExploreNYM/self-hosted-monitor/blob/main/nginx-certbot.sh) script and run it to setup Nginx and Certbot +```sh +chmod +x ~/self-hosted-monitor/nginx-certbot.sh && ~/self-hosted-monitor/nginx-certbot.sh +``` + +8. Give permissions to [`prometheus-target.sh`](https://github.com/ExploreNYM/self-hosted-monitor/blob/main/prometheus-target.sh) script and run it to add a scrape target. This can be run multiple times to add a new server to be monitored via Prometheus/ +```sh +chmod +x ~/self-hosted-monitor/prometheus-target.sh && ~/self-hosted-monitor/prometheus-target.sh +``` + +### The target server (the part to be monitored) setup + +If you run a monitoring stack and your node on two different VPS, the steps 9 and 10 need to be done on the VPS with the running node. In case you do it on the same VPS, skip directly to step 11 and continue. + +9. Install git +```sh +apt install git +``` + +10. Clone the repository to `~/self-hosted-monitor` +```sh +git clone https://github.com/ExploreNYM/self-hosted-monitor ~/self-hosted-monitor +``` +11. Give permissions to [`node-exporter.sh`](https://github.com/ExploreNYM/self-hosted-monitor/blob/main/node-exporter.sh) script and run it to setup Node exporter. +```sh +chmod +x ~/self-hosted-monitor/node-exporter.sh && ~/self-hosted-monitor/node-exporter.sh +``` + +### Grafana dashboard setup + +Finally we need to access Grafana dashboards. + +12. Open a browser at `http://:3000` or `https://` (depends on your setup), enter username `admin` and password `admin` and setup new credentials on prompt + +13. Setup *Data source* by opening menu -> `Connections` -> `Data sources` -> `+ Add new data source` -> `Prometheus` + +![](../images/grafana/add-data-sources.png) +![](../images/grafana/add-data-source-prometheus.png) + +14. In the field *Connection* next to `Prometheus server URL` enter `http://localhost:9090` (regardless if you accessing Grafana via `http` or `https` as this is for internal connection on the server). When you are done in the bottom confirm by `Save & Test` + +15. In the menu open: `Dashboards` -> `+ Create dashboard` -> `Import dashboard` + +![](../images/grafana/import-dashboard.png) + +16. ID field: enter `1860` -> `Load` + +![](../images/grafana/id-1860.png) + +17. In *Import dashboard* page select Prometheus in the bottom and finally `Import` + +![](../images/grafana/add-prometheus.png) + +Now you have your Prometheus panels displayed via Grafana dashboard for a simple monitoring of your node. + +## Verification and Troubleshooting + +To ensure that your services are running correctly, you can verify that by running `systemctl status ` or run a `journalctl -f -u ` to print service logs. It shall return status `Active: active (running)`. For example: +```sh +# to check if Prometheus service is active +systemctl status prometheus + +# to check if Grafana service is active +systemctl status grafana-server + +# to check if node-exporter service is active +systemctl status node_exporter + +# to run journal log +journalctl -f -u prometheus # or any other service you want to see +``` diff --git a/documentation/operators/src/testing/node-setup.md b/documentation/operators/src/testing/node-setup.md new file mode 100644 index 0000000000..3c1a1ca84c --- /dev/null +++ b/documentation/operators/src/testing/node-setup.md @@ -0,0 +1,79 @@ +# Node Setup for Performance Testing Event + +To join the [Performance testing event]({{performance_testing_webpage}}) node operators need to do proceed with the following tasks: + +1. **[Sign their node]({{performance_testing_webpage}}) into the testing environment** +2. **[Configure their node](#node-configuration) for the test** +3. (*Not mandatory*) [Setup metric monitoring system](templates.md) to observe node performance at any time + +## Node Configuration + +> Any syntax in `<>` brackets is a user's unique variable/version. Exchange with a corresponding name without the `<>` brackets. + +After you signed your node (or several) into the Performance testing environment, open the machine with (each of) your nodes and follow the steps below to configure your node for the event. + + +#### 1. Download and upgrade to `{{performance_testing_release}}` binary + - Download the binary from Nym [release page](https://github.com/nymtech/nym/releases/) (use `wget` or `curl` and binary download URL, don't compile from `master`) + - Follow the steps to upgrade node on the [maintenance page](../nodes/manual-upgrade.md) + - Before you re-start your node, follow the steps below + + +#### 2. If you run `nym-gateway` proceed with these steps. If not, go to the next point + - Make sure to have your `nym-gateway` setup as [Nym Exit Gateway](../legal/exit-gateway.md) following the commands [here](../nodes/gateway-setup.md#initialising-exit-gateway) + - Enable `[ip_packet_router]` (IPR) in your `~/.nym/gateways/*/config/config.toml` and IPv4 and IPv6 with [this script](https://gist.github.com/tommyv1987/ccf6ca00ffb3d7e13192edda61bb2a77) by running the two command below +```sh +# command to enable IPR +./nym-gateway setup-ip-packet-router --id --enabled true + +# script to enable IPv4 and IPv6 +curl -o enable_networking_for_nym_nodes.sh https://gist.githubusercontent.com/tommyv1987/ccf6ca00ffb3d7e13192edda61bb2a77/raw/0840e1d2ee9949716c45655457d198607dfd3107/enable_networking_for_nym_nodes.sh -L && chmod u+x enable_networking_for_nym_nodes.sh && sudo ./enable_networking_for_nym_nodes.sh +``` + + + + +#### 3. Restart your node with root privileges + - Either in a root shell or with `sudo -E` command + - In case you run your node as a [`systemd` service](../nodes/maintenance.md#systemd) make sure to run `systemctl daemon-reload` before the service restart + +## Troubleshooting + +If you come to any errors during the setup see troubleshooting page related to [Mix Nodes](../nodes/troubleshooting.md#mix-nodes) and [Gateways](../nodes/troubleshooting.md#gateways--network-requesters). In case your issue isn't documented ask in our Element [Node Operators channel](https://matrix.to/#/#operators:nymtech.chat) or raise an [issue](https://github.com/nymtech/nym/issues) on Github. + diff --git a/documentation/operators/src/testing/performance.md b/documentation/operators/src/testing/performance.md new file mode 100644 index 0000000000..ec4108d313 --- /dev/null +++ b/documentation/operators/src/testing/performance.md @@ -0,0 +1,32 @@ +# Performance Testing + +> To configure your node for a testing event, visit [node setup page](node-setup.md). + +Nym Mixnet has been running on mainnet for quite some time. There is still work to be done in order for the network to meet its full potential - mass adoption of privacy through fully distributed Mixnet. + +Nym asks its decentralised community of operators to join a series of performance testing events in order to **increase the overall quality of the Mixnet**. The main takeaways of such event are: + +1. Understanding of the network behavior under full load + - How many mixnet client users can all active set entry gateways handle simultaneously? + - How much sustained IP traffic can a subset of mainnet nodes sustain? +2. Needed improvements of Nym Node binaries to improve the throughput on mainnet +3. Measurement of required machine specs +4. Raw data record +5. Increase quality of Nym Nodes +6. Show each operator a way to monitor their nodes in a distributed fashion + +## Performance Testing Work Flow + +* Nym runs a paralel network environment [validator.performance.nymte.ch]({{performance_validator}}) with a chain ID `perf` +* Operators of Nym Nodes (currently `nym-mixnode` and `nym-gateway`) join by following easy steps on [performance testing web page]({{performance_testing_webpage}}), including simplified node authentication signature (while keep running their nodes on the mainnet) +* Once signed in, operators will be asked to swap their binary for the modified version with metrics endpoint to be able to connect their own [monitoring system](templates.md) +* Core node data will be fed to a unique mixnet contract for the `perf` side chain +* Nym starts a new API and start packet transition in high load through these nodes in both settings +* Nym tracks packet flow using Prometheus and Grafana +* Nym creates a large number of clients to the [performance validator network]({{performance_validator}}), intensifying the packet traffic +* Observe and record the metrics - this will probably put more strain on mainnet as well as many nodes spec is minimal + +## More Information + +* What happens after the test or what operators get for participating is shared up to date on the [performance testing web page]({{performance_testing_webpage}}) +* Visit our guides to [setup metrics template](templates.md) and learn how to operate them in self-custodial way diff --git a/documentation/operators/src/testing/prometheus-grafana.md b/documentation/operators/src/testing/prometheus-grafana.md new file mode 100644 index 0000000000..bd9f5af825 --- /dev/null +++ b/documentation/operators/src/testing/prometheus-grafana.md @@ -0,0 +1,36 @@ +# Prometheus & Grafana + +The combination of Prometheus and Grafana is a common stack used by Nym team for internal monitoring as well as by core community operators like [ExploreNym](https://github.com/ExploreNYM/vps-monitor) or [No Trust Verify](https://status.notrustverify.ch/d/CW3L7dVVk/nym-mixnet?orgId=1). + + + +## Prometheus + +[Prometheus](https://prometheus.io) is a free and open source monitoring systems. It allows operators to have their metrics, events, and alerts under full control. This ecosystem offers multiple advantages: + +- collects and records metrics from servers, containers, and applications +- provides a [flexible query language (PromQL)](https://prometheus.io/docs/prometheus/latest/querying/basics/) +- multiple modes visualization tools +- an alerting mechanism that sends notifications + +Prometheus collects and stores its metrics as time series data, i.e. metrics information is stored with the timestamp at which it was recorded, alongside optional key-value pairs called labels. + +## Grafana + +[Grafana](https://grafana.com/docs/grafana/latest/) is an open-source analytics and interactive front end. It is widely used for its easy to manage dashboards with visualizations like graphs, charts and alerts, all connected to live data sources. + +## Setup Guides + +There are various ways how to setup this stack. You can chose based on your preferences to do your own flow or follow some of the documented ones: + +- [ExploreNYM scripts](explorenym-scripts.md) for self-hosted monitoring +- Setup monitoring in a Docker container (*detailed guide will be published soon*) + + +## References and further reading + +* [Prometheus release page](https://prometheus.io/download/) +* [Prometheus documentation](https://prometheus.io/docs/introduction/overview/) +* Installation [guide to install Prometheus](https://www.cherryservers.com/blog/install-prometheus-ubuntu) on Ubuntu by cherryservers +* [Grafana installation guide](https://grafana.com/docs/grafana/latest/setup-grafana/installation/debian/) +* Nym's script [`prom_targets.py`](https://github.com/nymtech/nym/blob/promethus-is-our-friend/scripts/prom_targets.py) - a python program to request data from API and can be plugged to this stack diff --git a/documentation/operators/src/testing/templates.md b/documentation/operators/src/testing/templates.md new file mode 100644 index 0000000000..5bc922163f --- /dev/null +++ b/documentation/operators/src/testing/templates.md @@ -0,0 +1,17 @@ + Metrics of Performance Testing + +At Nym as well as several core community operators had setup metrics monitors for a clear overview of node performance. + +It is beneficial for a one time event or a limited period to connect nodes to the existing monitoring infrastructure system that Nym developers built to collect useful metrics. For the purpose of maximal privacy and decentralisation of the data - preventing Nym Mixnet from any global adversary takeover - we created these pages as a source of mutual empowerment, a place where operators can share and learn new skills to **setup metrics monitors on their own infrastructure**. + +## Collecting Testing Metrics + +For the purpose of the performance testing Nym core developers plan to run instances of Prometheus and Grafana connected to Node explorer in the house. The network overall key insights we seek from these tests are primarily internal. We're focused on pinpointing bottlenecks, capacity loads, and monitoring cpu usage on the nodes' machines. + +## Guides to Setup Own Metrics + +A list of different scripts, templates and guides for easier navigation: + +* [Prometheus and Grafana](prometheus-grafana.md) self-hosted setup +* [Nym-node CPU cron service](https://gist.github.com/tommyv1987/97e939a7adf491333d686a8eaa68d4bd) - an easy bash script by Nym core developer [@tommy1987](https://gist.github.com/tommyv1987), designed to monitor a CPU usage of your node, running locally +* Nym's script [`prom_targets.py`](https://github.com/nymtech/nym/blob/develop/scripts/prom_targets.py) - a useful python program to request data from API and can be run on its own or plugged to more sophisticated flows