Bug: Docker containers sometimes become inaccessible when using "network_mode:" via gluetun VPN container #405
Comments
|
|
Thanks for the update @qdm12 Answers:
I just upgraded gluetun container to the latest release and both containers are running https://github.com/qdm12/gluetun/releases/tag/v3.15.0 Ran the command I will update when the issue happens again, soon. |
|
Note:- None of the connected containers to gluetun are accessible, restarted one of them manually and it's working now.
Example, of the logs when ISP is down, below:
Question: |
|
Gluetun does not restart even if it loses connection. It will change to an unhealthy state though. You can see it received a termination signal before restarting: Maybe do you have it configured to restart when unhealthy? If so, that will break things up. You could have a script restart gluetun and then the other containers connected to it if it's unhealthy, until I address #386 |
|
You are about the termination check, I misworded that. Thanks for correcting that. Here is the stack I have for gluetun, fyi I still need to research how to restart the container and the other connected container on a health check. |
So to be clear the gluetun container (not openvpn inside) never restarts by itself right? It only restart when it's told to right? Also maybe try with docker compose version '3' maybe that helps for the networking part between containers.
I would advise you not to, I'll code that auto-healing in the coming days/2 weeks, it shouldn't be too hard to develop. |
I think yeah, as the docker logs not showing anything that it's restarting (Unless I do it manually), based on the Telegram notifier, it shows the following gluetun events happen from time to time when the ISP dropping, which is the termination part you mentioned above and this is normal.
That would be great!! 🥇 |
|
@qdm12 I have been monitoring the behavior of the gluetun container, and noticed something. It's actually restarting the container per Portainer run time and here is the logs Exit Code:
And then the connected containers lose the connection for sure since it's main network mode container was rebooting. And ISP wasn't down, everything network/internet and PIA were working fine. Is that expected behavior? |
|
|
|
@qdm12 makes sense. will try to reach out to Portainer to troubleshoot the root cause of that issue. I am not sure if it's Portainer that I have to talk with. As this issue happens between gluetun and docker daemon only. All the other containers are fine. Do recommend any logs to check/grab? |
|
Maybe there is something interesting in You could try running gluetun outside Portainer using docker-compose 3 in the cli, see if it gets restarted over time? |
|
Hi there, did you find the root cause in the end? Or got some logs/healthcheck logs? Cheers |
|
Hey @qdm12 no luck, I have been digging around with no luck, been doing manual restarted for qBittorrent manually whenever gluetun break and restarted by itself. |
|
Maybe out of the topic, but it will now restart Openvpn from within if it gets unhealthy. You can try by pulling the latest image. You should thus disable the auto healing now. |
|
@qdm12 I did update both official image and the testing one Since then, I haven't got any restarts on the container level My docker compose is the same version: '2.4'
services:
gluetun:
image: qmcgaw/gluetun
container_name: gluetun
environment:
- PUID=1029
- PGID=100
- TZ=America/Toronto
- VPNSP=private internet access
- REGION=CA Ontario
- PORT_FORWARDING=on #Complete https://github.com/qdm12/gluetun/wiki/Environment-variables
- PORT_FORWARDING_STATUS_FILE=/gluetun/port-forwarding/port.conf
- OPENVPN_USER=################# #Change to YOUR Username
- OPENVPN_PASSWORD=################ #Change to YOUR Password
volumes:
- /volume1/docker/gluetun/config:/gluetun
ports:
- 8000:8000 #HTTP Server https://github.com/qdm12/gluetun/wiki/HTTP-Control-server#OpenVPN
#- 19999:19999 #Netdata
- 666:80 #heimdal-VPN
- 4466:443 #heimdal-VPN
- 9080:9080 #QBitTorrent Web-UI
- 6881:6881 #QBitTorrent
- 6881:6881/udp #QBitTorrent
#- 9117:9117 #Jackett
#- 7878:7878 #Radarr
#- 8989:8989 #Sonarr
cap_add:
- NET_ADMIN
restart: unless-stoppedDo I need to add/remove something from the environment variables from above to disable anywhere the auto-heal? I think the change you did made a huge difference as a great workaround from the limitation of docker when container uses another container network and it doesn't break connectivity, since the network container doesn't restart anymore on the host level. Another note: Is this related you think? or maybe should be another issue # last Monday at 4:24:48 PM Running version latest built on 2021-04-19T19:54:17Z (commit fb8279f) I see in portainer it has health label though
|
|
No that's just for the vpn server side port forwarding which didn't answer within 30 seconds, probably a problem on PIA server side I'd say. Closing the issue then thanks! |
|
This sounds like a known issue that is identified in this video... |
Not that easy, but it's possible I believe. I'm working on it through github.com/qdm12/deunhealth |
|
Ive had ths problem but it seemed to because the ports either 80 or 443 are port forwarded to another IP address |
|
This is not fixed yet. Every couple of days I have to manually redeploy my *arr stacks which are connected to gluetun network. |
|
Interestingly, this problem caught with me once I started to use deunhealth. Before, I used some other autoheal which did not reacted instantly on "unhealthy" status, like deunhealth does using Docker events, and never had this problem. I guess gluetun restarts quickly, between two polls, and the old autoheal never caught it in unhealthy state. In fact, in my case (just checked), gluetun restarts and achieves "INFO [healthcheck] healthy!" in less than 4 seconds, while the default polling interval for old autoheal was 5 seconds. 😊 So, @qdm12 congrats! You made both gluetun and deunhealth to function a little bit too good. 😂 Now, there's a question on how to delay restart with deunhelth to allow container time to autoheal? @qdm12, would you be so kind to introduce a small variation to the label triggering deunhealth because vast majority of users have ISPs that change IP periodically by dropping the connection. It would be nice if it worked like this: deunhealth.restart.on.unhealthy=5000 Where 5000 is optional desired delay in miliseconds. |
|
@qdm12 this seems to still be an issue, happens to me pretty regularly, too. Let me know if I can provide additional info that could be of use |
Is this urgent?:
Yes#################################################################################
Host OS
CPU arch or device name:
Intel amd64What VPN provider are you using:
PIAWhat are you using to run your container?:
Docker ComposeWhat is the version of the program
#################################################################################
What's the problem 🤔
Docker containers become inaccessible when using "network_mode:" via gluetun VPN containerNote: Maybe this RFE I already created for some time would fix the issue?
#386
The only way to resolve the issue, is by manual restart for the container
#################################################################################
**log
For example, last logs for netdata:
For example, last logs for qBittorrent:
qt.network.ssl: QSslSocket::startClientEncryption: cannot start handshake on non-plain connection#################################################################################
### Notes:
I already created github bugs for those two products, but it only happens when I use the network mode with gluetun container, so I thought would be better to create the bug here.
netdata
netdata/netdata#10764
qBittorrent
linuxserver/docker-qbittorrent#105
You can find in both docker compose stack I used for each and troubleshooting steps taken.
Thanks
The text was updated successfully, but these errors were encountered: