Protonvpn: port forwarding connection timeout

[sruffilli]

Is this urgent?

None

Host OS

Ubuntu 22.10

CPU arch

x86_64

VPN service provider

ProtonVPN

What are you using to run the container

docker-compose

What is the version of Gluetun

Running version latest built on 2023-07-18T15:57:47.027Z (commit abe2ace)

What's the problem 🤔

It looks like querying to get the port-forwarding port always fail for me with a timeout error.
I tried multiple P2P cities to no avail.

Share your logs

gluetun-protonvpn  | 2023-07-21T14:32:23Z INFO [openvpn] [node-ch-06.protonvpn.net] Peer Connection Initiated with [AF_INET]185.159.157.129:443
gluetun-protonvpn  | 2023-07-21T14:32:23Z INFO [openvpn] setsockopt TCP_NODELAY=1 failed
gluetun-protonvpn  | 2023-07-21T14:32:23Z INFO [openvpn] TUN/TAP device tun0 opened
gluetun-protonvpn  | 2023-07-21T14:32:23Z INFO [openvpn] /sbin/ip link set dev tun0 up mtu 1500
gluetun-protonvpn  | 2023-07-21T14:32:24Z INFO [openvpn] /sbin/ip link set dev tun0 up
gluetun-protonvpn  | 2023-07-21T14:32:24Z INFO [openvpn] /sbin/ip addr add dev tun0 10.31.0.11/16
gluetun-protonvpn  | 2023-07-21T14:32:24Z INFO [openvpn] UID set to nonrootuser
gluetun-protonvpn  | 2023-07-21T14:32:24Z INFO [openvpn] Initialization Sequence Completed
gluetun-protonvpn  | 2023-07-21T14:32:24Z INFO [dns over tls] downloading DNS over TLS cryptographic files
gluetun-protonvpn  | 2023-07-21T14:32:24Z INFO [healthcheck] healthy!
gluetun-protonvpn  | 2023-07-21T14:32:25Z INFO [dns over tls] downloading hostnames and IP block lists
gluetun-protonvpn  | 2023-07-21T14:32:32Z INFO [healthcheck] unhealthy: dialing: dial tcp4: lookup cloudflare.com: i/o timeout
gluetun-protonvpn  | 2023-07-21T14:32:33Z INFO [dns over tls] init module 0: validator
gluetun-protonvpn  | 2023-07-21T14:32:33Z INFO [dns over tls] init module 1: iterator
gluetun-protonvpn  | 2023-07-21T14:32:34Z INFO [dns over tls] start of service (unbound 1.17.1).
gluetun-protonvpn  | 2023-07-21T14:32:34Z INFO [dns over tls] generate keytag query _ta-4a5c-4f66. NULL IN
gluetun-protonvpn  | 2023-07-21T14:32:34Z INFO [dns over tls] generate keytag query _ta-4a5c-4f66. NULL IN
gluetun-protonvpn  | 2023-07-21T14:32:34Z INFO [dns over tls] ready
gluetun-protonvpn  | 2023-07-21T14:32:34Z INFO [healthcheck] healthy!
gluetun-protonvpn  | 2023-07-21T14:32:34Z INFO [vpn] You are running 1 commit behind the most recent latest
gluetun-protonvpn  | 2023-07-21T14:32:34Z INFO [ip getter] Public IP address is 185.159.157.21 (Switzerland, Zurich, Zürich)
gluetun-protonvpn  | 2023-07-21T14:32:34Z INFO [vpn] VPN gateway IP address: 10.31.0.1
gluetun-protonvpn  | 2023-07-21T14:34:42Z ERROR [port forwarding] getting external IPv4 address: executing remote procedure call: connection timeout: after 2m7.75s
gluetun-protonvpn  | 2023-07-21T14:34:42Z INFO [port forwarding] retrying in 5s
gluetun-protonvpn  | 2023-07-21T14:36:55Z ERROR [port forwarding] getting external IPv4 address: executing remote procedure call: connection timeout: after 2m7.75s
gluetun-protonvpn  | 2023-07-21T14:36:55Z INFO [port forwarding] retrying in 5s
gluetun-protonvpn  | 2023-07-21T14:39:08Z ERROR [port forwarding] getting external IPv4 address: executing remote procedure call: connection timeout: after 2m7.75s
gluetun-protonvpn  | 2023-07-21T14:39:08Z INFO [port forwarding] retrying in 5s

Share your configuration

services:
  gluetun-protonvpn:
    image: qmcgaw/gluetun:latest
    container_name: gluetun-protonvpn
    cap_add:
      - NET_ADMIN
    devices:
      - /dev/net/tun:/dev/net/tun
    volumes:
      - ${CONFIGS_PATH}/gluetun:/tmp/gluetun
    ports:
      - [...]
    environment:
      - VPN_SERVICE_PROVIDER=protonvpn
      - VPN_TYPE=openvpn      
      - OPENVPN_USER=<user>+pmp
      - OPENVPN_PASSWORD=<pass>
      - SERVER_COUNTRIES={Switzerland, France, Italy, Spain}
      - FIREWALL_OUTBOUND_SUBNETS=10.0.0.0/8,172.16.0.0/12,192.168.0.0/16
      - VPN_ENDPOINT_PORT=443
      - VPN_PORT_FORWARDING=on
      - TZ=Etc/UTC
    restart: always

[sruffilli]

I was able to self-diagnose the issue.
When FIREWALL_OUTBOUND_SUBNETS overlaps with the IP address assigned to the TUN interface (which was on the 10/8 range), the port_forwarding check fails. I simply removed 10/8 from FIREWALL_OUTBOUND_SUBNETS in the configuration (as I don't actually need the whole RFC1918), which did the trick.

@qdm12 could this be moved to a docubug?

[qdm12]

@sruffilli Awesome you self debugged it 👍 I probably would had mislead you with other tries 😄

Although instead of documenting it, it might be interesting to either:

1. IP rules so the vpn gateway ip is routed through the tunnel interface and has priority over the outbound subnets; or
2. Remove the vpn gateway ip from the outbound subnets routing

Although both are a bit 'dark magic' and could be confusing if you try to access a lan ip with the same ip as your vpn gateway ip. A warning could be logged but still not totally ideal. Thoughts?

@ver151set please create another issue with your full logs 🙏 Thanks! (edit: never mind, you already done it, I'll hide your comment to keep this issue clean)

[sruffilli]

1. IP rules so the vpn gateway ip is routed through the tunnel interface and has priority over the outbound subnets; or
2. Remove the vpn gateway ip from the outbound subnets routing

I like 1 very much (and I'm not sure how you'd implement 2 tbh).

Although both are a bit 'dark magic' and could be confusing if you try to access a lan ip with the same ip as your vpn gateway ip. A warning could be logged but still not totally ideal. Thoughts?

Overlapping ranges always require a some arcane art tinkering doesn't it :)?
The only solution I could think of would be layering an additional DNAT+SNAT to all outbound traffic, but a) I'm sure it'd introduce additional issues and b) I don't know how it would interact with existing rules.
+1 for logging.

[qdm12]

I think having an ip rule to route that single gateway ip address through the tunnel and leave the rest of the outbound subnet working + log a warning about it is the best way to do it.

Now before I jump to implement this, can you try in a running gluetun on your side:

docker exec gluetun ip route add 10.31.0.1/32 dev tun0

and check if this resolves it? This should route traffic towards 10.31.0.1 through the tun0 vpn network interface.
If it does fix it, I'll code it so it's automated.

[lollenderrofler]

Without changing anything I am now also getting this issue. It was working before. I've set FIREWALL_OUTBOUND_SUBNETS=192.168.188.0/24 and nothing more, Gluetun picks

2023-07-28T22:51:45+02:00 INFO [routing] default route found: interface eth0, gateway 192.168.80.1, assigned IP 192.168.80.2 and family v4

and the logs say

2023-07-28T22:52:04+02:00 INFO [vpn] You are running on the bleeding edge of latest!

2023-07-28T22:52:04+02:00 INFO [vpn] VPN gateway IP address: 10.2.0.1

2023-07-28T22:52:04+02:00 INFO [port forwarding] gateway external IPv4 address is 190.2.146.180

2023-07-28T22:52:04+02:00 INFO [ip getter] Public IP address is 190.2.146.230 (Netherlands, North Holland, Amsterdam)

2023-07-28T22:54:11+02:00 ERROR [port forwarding] adding port mapping: executing remote procedure call: connection timeout: after 2m7.75s

2023-07-28T22:54:11+02:00 INFO [port forwarding] retrying in 5s

If you need more info on my setup, for easier reading please see my other issue #1749

Thank you for looking into all these issues and making such a great product!

[sruffilli]

can you try in a running gluetun on your side:

docker exec gluetun ip route add 10.31.0.1/32 dev tun0

and check if this resolves it?

I tested it and it doesn't - I suspect the reason is that 99 is taking precedence over the route you suggested.

/ # ip rule
0:	from all lookup local
98:	from all to 172.20.0.0/24 lookup main
99:	from all to 172.16.0.0/12 lookup 199
99:	from all to 192.168.0.0/16 lookup 199
99:	from all to 10.0.0.0/8 lookup 199
100:	from 172.20.0.4 lookup 200
32766:	from all lookup main
32767:	from all lookup default

Haven't had time to test something different - will do and report back.

[qdm12]

I tested it and it doesn't - I suspect the reason is that 99 is taking precedence over the route you suggested.

Oh yes that's why, but do you actually need 192.168.0.0/16 in your firewall outbound subnets?? Related which may fix this is #1697