Bad Gateway when trying to access docker service with Proxmox

[tobiraak]

I have a vps, setup everything as in the documentation stated and connected my NAS as a site successfully by installing newt on my proxmox debian vm. Now I have a service (deployed with docker compose) locally accessible on 192.168.178.104:9090 on that vm. Entering these values when creating a ressource will result in the Gateway timeout error. Do I have to configure anything in proxmox? Any help would be appreciated. Do I have to configure newt in the same compose file as my service and put them in a network?

Originally posted by @tobiraak in #302

[hhftechnology]

I have a vps, setup everything as in the documentation stated and connected my NAS as a site successfully by installing newt on my proxmox debian vm. Now I have a service (deployed with docker compose) locally accessible on 192.168.178.104:9090 on that vm. Entering these values when creating a ressource will result in the Gateway timeout error. Do I have to configure anything in proxmox? Any help would be appreciated. Do I have to configure newt in the same compose file as my service and put them in a network?

Originally posted by @tobiraak in #302

Just an example.

This is a small snippet from my write up. this is how i handle newt.

Edit /etc/netplan/00-installer-config.yaml:

network:
  version: 2
  ethernets:
    ens18:
      addresses:
        - 192.168.20.10/24
      gateway4: 192.168.20.1
      nameservers:
        addresses: [1.1.1.1, 8.8.8.8]

Apply with:

sudo netplan apply

4. Setting Up Docker with Network Isolation

Install Docker and create isolated networks:

# Install Docker
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh

# Create dedicated networks for exposed applications
docker network create --subnet=172.20.0.0/16 exposed-apps

Implementing Newt in a Confined Environment

Newt will run in a Docker container, isolating it from the host system and only allowing it to communicate with specific application containers.

1. Setting Up Newt Container

Create a dedicated directory for Newt:

mkdir -p /opt/newt
cd /opt/newt

Create a docker-compose.yml file:

networks:
  exposed-apps:
    external: true

services:
  newt:
    container_name: newt-client
    image: fosrl/newt:latest
    restart: unless-stopped
    environment:
      - PANGOLIN_ENDPOINT=https://pangolin.yourdomain.com
      - NEWT_ID=your_newt_id_from_pangolin
      - NEWT_SECRET=your_newt_secret_from_pangolin
    networks:
      exposed-apps:
        ipv4_address: 172.20.0.2
    cap_add:
      - NET_ADMIN
    volumes:
      - ./logs:/var/log/newt

2. Getting Configuration from Pangolin

To obtain the Newt ID and secret, you need to:

1. Log into your Pangolin dashboard on the VPS
2. Create a new organization if you don't have one
3. Create a new site (select "Newt" as the connection method)
4. Copy the generated ID and secret
5. Update your docker-compose.yml with these values

3. Starting and Securing Newt

Start the Newt container:

docker compose up -d

Check that it's running and connecting correctly:

docker logs -f newt-client

You should see messages about establishing a connection to Pangolin and creating a WireGuard interface. If everything is working, the site status in Pangolin should change to "Online".

Connecting Exposed Applications

Now we'll set up the applications that should be exposed to the internet. We'll create a separate Docker Compose file for each application, ensuring they're isolated but accessible through Newt.

1. Example: Setting Up Nextcloud

Create directory:

mkdir -p /opt/nextcloud
cd /opt/nextcloud

Create docker-compose.yml:

networks:
  exposed-apps:
    external: true
  nextcloud-internal:
    internal: true

services:
  nextcloud-db:
    image: mariadb:10.6
    container_name: nextcloud-db
    restart: unless-stopped
    command: --transaction-isolation=READ-COMMITTED --log-bin=binlog --binlog-format=ROW
    networks:
      - nextcloud-internal
    volumes:
      - ./nextcloud-db:/var/lib/mysql
    environment:
      - MYSQL_ROOT_PASSWORD=secure_root_password
      - MYSQL_PASSWORD=secure_nextcloud_password
      - MYSQL_DATABASE=nextcloud
      - MYSQL_USER=nextcloud
    
  nextcloud-app:
    image: nextcloud:latest
    container_name: nextcloud-app
    restart: unless-stopped
    depends_on:
      - nextcloud-db
    networks:
      nextcloud-internal:
      exposed-apps:
        ipv4_address: 172.20.0.10
    volumes:
      - ./nextcloud-data:/var/www/html
    environment:
      - MYSQL_HOST=nextcloud-db
      - MYSQL_PASSWORD=secure_nextcloud_password
      - MYSQL_DATABASE=nextcloud
      - MYSQL_USER=nextcloud
      - NEXTCLOUD_TRUSTED_DOMAINS=nextcloud.yourdomain.com
      - OVERWRITEPROTOCOL=https

Start Nextcloud:

docker compose up -d

thumb rule is newt should be on the same network as the services it wants to expose to pangolin on VPS.

[tobiraak]

I'm sorry but I can't seem to follow your guide. Where do you specify the port? I also do not have a netplan directory.
Which IP address do you specify as a ressource in the pangolin dashboard? Thanks in advance!

[tobiraak]

Update: This seems to be a problem with a proxmox VM. I am running a debian VM in proxmox and I can't seem to find out why it would fail. The IP of my vm is reachable in my network but seems to make problems since it is just a VM. Does anyone know if I need to configure the proxmox host in order to make it work?

[tobiraak]

Another update: I have ufw enabled on my VM of course. Disabling it made it suddenly work. If I add the port of the service that I want to expose to the ufw rules, it is working. But this is not the intended behavior, since it is tunneling it anyway and I should not need to open any ports. So the question remains, do I need to update any proxmox network settings or enable any kind of port forwarding?
I have no clue why I need to open ports in the VM but if I use newt and pango with my raspberry, I don't have to open any ports at all.

[timmish]

@tobiraak
did you solve this?
I have the same issue. When using UFW, I have to allow the port I am forwarding and tunneling with Pangolin - this should not be the case, but I don't now what to do.

[tobiraak]

No I've not solved it, I have UFW disabled completely and it sucks since this issue was closed without beeing solved or explained.

[timmish]

Yes 🥲
Did you switch to iptables or another firewall? No firewall is no solution for me I guess.

[tobiraak]

Try the Firewall option from proxmox, this might work? I ditched this solution tbh because of this. Maybe I should ask this question on reddit.com/r/selfhosted