docker-compose.yml for mempool

[N8schrath]

I'm asking for help rather than raising an issue.

I'm running my own mempool instance with docker-compose, as shown on github.com/mempool/mempool and github.com/mempool/mempool/tree/master/docker. If UFW is disabled everything works as expected, but with UFW enabled the known issues occur. I installed and prepared everything along your instructions, but the problem is I don't know which edits I have to make in "docker-compose.yml", because I know way to little to do this on my own even with the examples you showed.
This is the file:

version: "3.7"

services:
  web:
    environment:
      FRONTEND_HTTP_PORT: "8080"
      BACKEND_MAINNET_HTTP_HOST: "api"
    image: mempool/frontend:latest
    user: "1000:1000"
    restart: always
    stop_grace_period: 1m
    command: "./wait-for db:3306 --timeout=720 -- nginx -g 'daemon off;'"
    ports:
      - 4080:8080

  api:
    environment:
      MEMPOOL_BACKEND: "electrum"
      ELECTRUM_HOST: "10.0.0.9"
      ELECTRUM_PORT: "50002"
      ELECTRUM_TLS_ENABLED: "true"
      CORE_RPC_HOST: "10.0.0.9"
      CORE_RPC_PORT: "8332"
      CORE_RPC_USERNAME: "myusername"
      CORE_RPC_PASSWORD: "mypassword"
      DATABASE_ENABLED: "true"
      DATABASE_HOST: "db"
      DATABASE_DATABASE: "mempool"
      DATABASE_USERNAME: "mempool"
      DATABASE_PASSWORD: "mempool"
      STATISTICS_ENABLED: "true"
    image: mempool/backend:latest
    user: "1000:1000"
    restart: always
    stop_grace_period: 1m
    command: "./wait-for-it.sh db:3306 --timeout=720 --strict -- ./start.sh"
    volumes:
      - ./data:/backend/cache
  db:
    environment:
      MYSQL_DATABASE: "mempool"
      MYSQL_USER: "mempool"
      MYSQL_PASSWORD: "mempool"
      MYSQL_ROOT_PASSWORD: "admin"
    image: mariadb:10.5.8
    user: "1000:1000"
    restart: always
    stop_grace_period: 1m
    volumes:
      - ./mysql/data:/var/lib/mysql

I understand that I have to add something like this:

    labels:
      UFW_MANAGED: 'TRUE'
      UFW_ALLOW_FROM: '172.10.50.32;192.168.3.0/24;10.10.0.50/32-8080-LAN'
      UFW_DENY_OUT: 'TRUE'
      UFW_ALLOW_TO: '8.8.8.8-53-GoogleDNS;1.1.1.0/24-53-CloudflareDNS;192.168.10.24-8080-LAN'
    networks:
      - my-network

But I don't know in which way I have to edit the values from the example to fit to my "docker-compose.yml"-File.

[shinebayar-g]

Hi. What exactly are you trying to accomplish?
I see you have

    ports:
      - 4080:8080

for your web: service. Do you want to protect this container with ufw?
If that's the case you add the labels to your web: service. Looks like other containers don't need any ufw labels because they don't have any ports published.

[shinebayar-g]

Okay. I understand that you're trying to open web service to your browser? In that case you have a few options.

1. If you don't have to open web service externally. For example only localhost is fine, then you don't need ufw at all. Just do

ports:
      - 127.0.0.1:4080:8080

This makes container's 8080 port available to your localhost:4080 or 127.0.0.1:4080

2. If you need to open web service externally, for example both to localhost and other containers in the same docker compose network: You'll need same ports defined as step 1. But containers in the same network should refer each other by its name. like web:8080, api:50002 etc. You most likely won't need ufw at all in this scenario. Unless you want to be really restricted. Bonus point: Make sure how your browser is supposed to connect to each service. For example, if it needs to connect to both web and api in that case you have to publish ports on both etc.

3. If you need to open your containers externally beyond your localhost and docker compose network, for example to another container in the different network or another device in your LAN network or even Direct Internet connection etc.. Then you can make use of ufw.
   For example, assuming you're going to open web and api to some random IP address on your network. Then the syntax would be:

version: "3.7"

services:
  web:
    environment:
      FRONTEND_HTTP_PORT: "8080"
      BACKEND_MAINNET_HTTP_HOST: "api"
    image: mempool/frontend:latest
    user: "1000:1000"
    restart: always
    stop_grace_period: 1m
    command: "./wait-for db:3306 --timeout=720 -- nginx -g 'daemon off;'"
    ports:
      - 4080:8080
    # By adding these labels you're adding ufw rules that allow connections from the specified IP addresses to this specific container.
    # check ufw rules after starting the container with: sudo ufw status
    # Notice that you're specifying the port of the container, not the host port.
    labels:
      UFW_MANAGED: 'TRUE'
      UFW_ALLOW_FROM: '192.168.50.132-8080-SomeRandomDeviceOnYourLan;192.168.0.0/24-OrYourEntireLan'

  api:
    environment:
      MEMPOOL_BACKEND: "electrum"
      ELECTRUM_HOST: "10.0.0.9"
      ELECTRUM_PORT: "50002"
      ELECTRUM_TLS_ENABLED: "true"
      CORE_RPC_HOST: "10.0.0.9"
      CORE_RPC_PORT: "8332"
      CORE_RPC_USERNAME: "myusername"
      CORE_RPC_PASSWORD: "mypassword"
      DATABASE_ENABLED: "true"
      DATABASE_HOST: "db"
      DATABASE_DATABASE: "mempool"
      DATABASE_USERNAME: "mempool"
      DATABASE_PASSWORD: "mempool"
      STATISTICS_ENABLED: "true"
    image: mempool/backend:latest
    user: "1000:1000"
    restart: always
    stop_grace_period: 1m
    command: "./wait-for-it.sh db:3306 --timeout=720 --strict -- ./start.sh"
    volumes:
      - ./data:/backend/cache

  db:
    environment:
      MYSQL_DATABASE: "mempool"
      MYSQL_USER: "mempool"
      MYSQL_PASSWORD: "mempool"
      MYSQL_ROOT_PASSWORD: "admin"
    image: mariadb:10.5.8
    user: "1000:1000"
    restart: always
    stop_grace_period: 1m
    volumes:
      - ./mysql/data:/var/lib/mysql

[N8schrath]

Thank you shinebayar-g for your detailed answer. Unfortunately I'm all too busy these days so I did not yet have time to try everything you suggested. But as soon as possible I will do that and post an update how it all worked out, as this problem is nagging me and I need to get it right.

[N8schrath]

I did try all your suggestions, as far as I understood and to my best effort, but to no avail.

Perhaps it helps if I add or reiterate some information on the faulty behavior I need to correct:

• If I run docker-compose up with UFW disabled there's no problem at all, everything works as expected (settings of docker-compose.yml are as I originally posted)

• If I then enable UFW (or docker-compose up with already enabled UFW) mempool doesn't load data in the webbrowser-tab http://127.0.0.1:4080/ and I get the following errors from the docker api-container:

  • docker-api-1 | Mar 6 13:23:40 [100] ERR: Electrum error: {"errno":-110,"code":"ETIMEDOUT","syscall":"connect","address":"10.0.0.9","port":50002}
  • docker-api-1 | Mar 6 13:23:40 [100] ERR: Error connecting to Electrum Server at 10.0.0.9:50002
  • docker-api-1 | Mar 6 13:23:40 [100] INFO: Disconnected from Electrum Server at 10.0.0.9:50002
  • docker-api-1 | Mar 6 13:25:35 [100] WARN: runMainLoop error: ETIMEDOUT. Retrying in 10 sec.

• So how I interpret this error, is that the docker api-container cannot get data from the electrum-server simply because it cannot connect to the electrum server (as I said, everything runs on the same machine, no external machines are involved in my setup)

So the remaining questions are:

• Why and how does UFW block the docker api-container from connecting with the electrum-server that runs on the same machine as docker-compose?
• Where exactly does this problem/conflict origin, as there is no external traffic to and from my machine and the internet (only the electrum-server gets data from the internet) and how to solve it?

[shinebayar-g]

I assume you already put ufw-docker rules from step 1 into your ufw rules and restarted ufw?

Please put full reproducible steps 1 by 1 if possible and what's the expected result. It's still unclear to me how your ufw is setup.

[N8schrath]

Hello shinebayar-g.
I was finally able to solve the problem. 😌

I tried adding UFW rules as shown at Dojo x86 Bitcoin Node Guide: 5. Whirlpool & UFW:
sudo ufw default deny incoming
sudo ufw default allow outgoing
sudo ufw allow 50002/tcp
sudo ufw enable

With sudo ufw allow 50002/tcp being the one that solved the issue I guess.

Thank you for your time and help, maybe this thread helps other to solve similar issues with UFW.