added loki logging configuration for api keypers

README.md

@@ -6,7 +6,7 @@ This repository contains the docker compose configuration to run a shutter api k
 
 ### Chain execution clients
 
-Keypers are required to have access to a the Chain's execution client API, where the shutter registry and keyperset manager contracts are deployed.
+Keypers are required to have access an execution client's JSON RPC API, of the chain, where the shutter registry and keyperset manager contracts are deployed.
 
 ### System requirements
 
@@ -27,14 +27,15 @@ Shutter Network does operate a prometheus compatible monitoring system. See belo
 
 ### Software
 
-- You will need a recent version of `docker` and the `docker compose` cli plugin. 
+- You will need a recent version of `docker` and the `docker compose` cli plugin.
 - For cloning the repository you will need `git`.
 
 The Keyper node is distributed as a docker-compose stack consisting of multiple services. This repository contains all necessary files.
 
-One component is opt-in monitoring, which by default requires opening a port for scrape access to the metrics endpoints, 
-as well as a public IP address that needs to be shared with the Shutter team if you wish to participate in system-wide 
+One component is opt-in monitoring, which by default requires opening a port for scrape access to the metrics endpoints,
+as well as a public IP address that needs to be shared with the Shutter team if you wish to participate in system-wide
 monitoring of the nodes.
+
 If you would rather not open up the metrics endpoints we also provide support for push based monitoring.
 
 See the `Metrics` section in the .env file for more information.
@@ -53,35 +54,65 @@ git checkout shutter-api
 ```
 
 2. Copy the `example-api.env`(*) file to `.env` and fill in your information:
-   - **Required values**
-     - Your Ethereum account key (hex-encoded *without* `0x` prefix): `SIGNING_KEY`
+  - **Required values**
+    - Your Ethereum account key (hex-encoded *without* `0x` prefix): `SIGNING_KEY`
 
-       **IMPORTANT**: Please double-check that you are using the key associated with the address that you provided during the Keyper application process. Otherwise, your Keyper node will not be able to join the network.
-     - A name of your choice for your keyper node: `KEYPER_NAME`
+      **IMPORTANT**: Please double-check that you are using the key associated with the address that you provided during the Keyper application process. Otherwise, your Keyper node will not be able to join the network.
+    - A name of your choice for your keyper node: `KEYPER_NAME`
 
-       (Please use only letters, numbers, and underscores. No spaces or special characters.)
-     - Your **public** IP address: `PUBLIC_IP`
+      (Please use only letters, numbers, and underscores. No spaces or special characters.)
+    - Your **public** IP address: `PUBLIC_IP`
 
-       It is important that this is the address your node is reachable under from the internet since it is used for the P2P network between the nodes.
-     - A Chain execution JSON RPC API endpoint (WebSocket): `CHAIN_EXECUTION_RPC_WS_URL`
-   - Metrics (optional):
-     - To enable metrics, set `METRICS_ENABLED` to `true` (the default)
-     - Define the interface the metrics ports (`:9200` and `:27660`) should be exposed on with `METRICS_INTERFACE` (defaults to `0.0.0.0`, i.e. the public interface)
-     - If you rather not publicly expose the metrics and would like to push metrics instead, uncomment the `COMPOSE_PROFILES=pushmetrics` line and set `METRICS_INTERFACE` to `127.0.0.1`.
-       - Define the target(s) for the pushgateway with `PUSHGATEWAY_URL` (multiple targets can be separated by commas).
-
-         The default value points to a pushgateway operated by the Shutter Network team. To gain access please ask for credentials in the Shutter Network Discourse forum.     
+      It is important that this is the address your node is reachable under from the internet since it is used for the P2P network between the nodes.
+    - A Chain execution JSON RPC API endpoint (WebSocket): `CHAIN_EXECUTION_RPC_WS_URL`
+  - Metrics (optional):
+    - To enable metrics, set `METRICS_ENABLED` to `true` (the default)
+    - Define the interface the metrics ports (`:9200` and `:27660`) should be exposed on with `METRICS_INTERFACE` (defaults to `0.0.0.0`, i.e. the public interface)
+    - If you rather not publicly expose the metrics and would like to push metrics instead, uncomment the `COMPOSE_PROFILES=pushmetrics` line and set `METRICS_INTERFACE` to `127.0.0.1`.
+      - Define the target(s) for the pushgateway with `PUSHGATEWAY_URL` (multiple targets can be separated by commas).
 
-> *) **NOTE**: The `example-mainnet.env` file is a template for Gnosis mainnet deployment. If you want to deploy a Keyper for the Chiado testnet instead, use the `example-chiado.env` file.
+        The default value points to a pushgateway operated by the Shutter Network team. To gain access please ask for credentials in the Shutter Network Discourse forum.
+  - Logging collection (optional):
+    - To push logs to loki/vmlogs server:
+      - Define the url for the server to push logs to, with `LOKI_URL`. The default value points to the logging server operated by the Shutter Network Team. To gain access please ask for credentials in the Shutter Network Discourse forum.
+      - Use the `docker-compose.loki.yml` file, which overrides logging, as shown under the running and update sections.
 
-## Running
+> **NOTE**: The `example-api.env` file is a template for the Shutter API Keyper deployment on Gnosis Mainnet.
 
-You start your Keyper node by running
+## Setting Up Logging
 
-```
+To enable log pushing to a **Loki/VmLogs server**, follow these steps:
+
+1. **Update `.env` to include Loki logs configuration using the credentials provided by the Shutter team:**
+   ```sh
+   LOKI_URL=https://<user_id>:<password>@logs.metrics.shutter.network/insert/loki/api/v1/push
+   ```
+2. **Install the Loki Docker driver (if not installed):**
+   ```sh
+   docker plugin install grafana/loki-docker-driver:3.3.2-amd64 --alias loki --grant-all-permissions
+   ```
+
+> **Note:** For **ARM64 hosts**, add `-arm64` to the image tag.
+
+For more details, refer to the [Docker driver client | Grafana Loki documentation](https://grafana.com/docs/loki/latest/send-data/docker-driver/).
+
+
+> **IMPORTANT:** If logging is enabled, make sure to use the correct `docker compose` commands as noted below.
+
+## Running the Keyper Node
+
+### **Without Pushing Logs**
+To start the Keyper node run:
+```sh
 docker compose up -d
 ```
 
+### **With Log Pushing Enabled**
+> **IMPORTANT:** Logging requires additional configuration steps. Follow the [Logging Setup](#setting-up-logging) section before running the following command:
+```sh
+docker compose -f docker-compose.yml -f docker-compose.loki.yml up -d
+```
+
 ## Backups
 
 Once your Keyper is up and running, you should regularly back up the following:
@@ -92,7 +123,9 @@ Once your Keyper is up and running, you should regularly back up the following:
 
 These files will allow you to re-build your Keyper in case of data loss.
 
-## Updating
+## **Updating**
+
+### **Without Pushing Logs**
 
 ```shell
 cd shutter-keyper-deployment
@@ -101,6 +134,12 @@ git checkout shutter-api/<new-version-tag>
 docker compose up -d
 ```
 
+### **Updating with Log Pushing Enabled**
+> **IMPORTANT:** If logging is enabled, ensure you follow the [Logging Setup](#setting-up-logging) section first. Then, use the correct command:
+```sh
+docker compose -f docker-compose.yml -f docker-compose.loki.yml up -d
+```
+
 ## Version History
 ### `shutter-api-keyper/2025.02.01`
 - initial public release
@@ -112,4 +151,4 @@ docker compose up -d
   RegistryContract: 0x7D18359c2f49e4aEBc0df761B1152c31DE044e83
   KeysetManagerContract: 0xFaB842d8Ff826E93D31f839AD47218d8c9511cC6
   KeyBroadcastContract: 0x474079EFa5D93bb48a1cA27Ebbf46d32E273A28b
-```
+```

docker-compose.loki.yml

@@ -0,0 +1,14 @@
+services:
+  chain:
+    logging: !override
+      driver: loki
+      options:
+        loki-url: ${LOKI_URL}
+        loki-external-labels: keyper_name=kpr-${KEYPER_NAME}
+
+  keyper:
+    logging: !override
+      driver: loki
+      options:
+        loki-url: ${LOKI_URL}
+        loki-external-labels: keyper_name=kpr-${KEYPER_NAME}

example-api.env

@@ -47,6 +47,24 @@ METRICS_LISTEN_IP=0.0.0.0
 # Replace <USERNAME> and <API_KEY> above with the credentials you receive.
 # Multiple targets and their credentials can be specified by separating them with a comma.
 
+## Push Logging ##
+
+# To enable push logs, loki docker driver needs to be installed first by following cmd.
+# `docker plugin install grafana/loki-docker-driver:3.3.2-amd64 --alias loki --grant-all-permissions`
+# Please change `amd64` to `arm64` based on machine service is hosted on
+
+#Uncomment to enable push logs
+# Target URL(s) for push logs in Loki remote write format
+#LOKI_URL=https://<user_id>:<password>@logs.metrics.shutter.network/insert/loki/api/v1/push
+
+# The default URL above points to a loki server by Shutter Network (which we
+# would appreciate you send logs to), please ask for an access key in the
+# Keyper section of the Shutter Network discourse forum.
+# Replace <user_id> and <password> above with the credentials you receive.
+
+# To run with push logs to loki, the sevice should be run by command:
+# `docker compose -f docker-compose.yml -f docker-compose.loki.yml up -d`
+
 
 ######################################
 ## These values MUST not be touched!