containers in bridge network can't access internet (no route to host) #2925
Description
Description
Few weeks ago, containers inside bridge network stopped to access internet.
Accessing a machine on lan is fine outside is not. However from containers not in bridge network and with the host it's still possbile to join internet.
- Create a bridge network
- Set a container inside it
- Try to ping / curl / whatever access a machine outside LAN
Containers nextcloud is in a bridged network but plex is not.
$ docker exec -it nextcloud /bin/bash
root@82002fe9f925:/var/www/html# curl http://172.217.19.238
curl: (7) Failed to connect to 172.217.19.238 port 80: No route to host
root@82002fe9f925:/var/www/html# curl http://192.168.1.22
<HTML><HEAD><TITLE>401 Unauthorized</TITLE></HEAD>
<BODY BGCOLOR="#cc9999"><H4>401 Unauthorized</H4>
Authorization required.
</BODY></HTML>
root@82002fe9f925:/var/www/html# exit
$ docker exec -it plex /bin/bash
root@nop:/# curl http://google.com
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
<TITLE>301 Moved</TITLE></HEAD><BODY>
<H1>301 Moved</H1>
The document has moved
<A HREF="http://www.google.com/">here</A>.
</BODY></HTML>
Describe the results you expected:
Should access internet within a bridged network.
Additional information you deem important (e.g. issue happens only occasionally):
If have of course troubleshooted dns and more but seem really linked to iptable. I do not have other firewall. I have rebooted, reinstalled several time docker. Restarted services and so on ..
Also my host has access to internet without issue.
Output of docker version:
Client: Docker Engine - Community
Version: 20.10.2
API version: 1.41
Go version: go1.13.15
Git commit: 2291f61
Built: Mon Dec 28 16:17:34 2020
OS/Arch: linux/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.2
API version: 1.41 (minimum version 1.12)
Go version: go1.13.15
Git commit: 8891c58
Built: Mon Dec 28 16:15:28 2020
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.4.3
GitCommit: 269548fa27e0089a8b8278fc4fc781d7f65a939b
runc:
Version: 1.0.0-rc92
GitCommit: ff819c7e9184c13b7c2607fe6c30ae19403a7aff
docker-init:
Version: 0.19.0
GitCommit: de40ad0
Output of docker info:
Client:
Context: default
Debug Mode: false
Plugins:
app: Docker App (Docker Inc., v0.9.1-beta3)
buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)
Server:
Containers: 14
Running: 13
Paused: 0
Stopped: 1
Images: 27
Server Version: 20.10.2
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 269548fa27e0089a8b8278fc4fc781d7f65a939b
runc version: ff819c7e9184c13b7c2607fe6c30ae19403a7aff
init version: de40ad0
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 5.9.0-0.bpo.2-amd64
Operating System: Debian GNU/Linux 10 (buster)
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 5.815GiB
Name: nop.whiterabbit
ID: RALF:C6QQ:HLAY:HWMD:4MJ6:4WRC:MLWD:JVNS:CUWE:TDF6:OPMI:P5E3
Docker Root Dir: /var/lib/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: API is accessible on http://0.0.0.0:2376 without encryption.
Access to the remote API is equivalent to root access on the host. Refer
to the 'Docker daemon attack surface' section in the documentation for
more information: https://docs.docker.com/engine/security/security/#docker-daemon-attack-surface
WARNING: No blkio weight support
WARNING: No blkio weight_device support
Additional environment details (AWS, VirtualBox, physical, etc.):
I'am using Open Media Vault 5 as OS
Linux nop.whiterabbit 5.9.0-0.bpo.2-amd64 #1 SMP Debian 5.9.6-1~bpo10+1 (2020-11-19) x86_64 GNU/Linux
IPTABLES OUTPUT
# Generated by xtables-save v1.8.2 on Sun Jan 10 00:31:59 2021
*nat
:PREROUTING ACCEPT [0:0]
:INPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A POSTROUTING -s 172.18.0.0/16 ! -o br-aa7ab3e7773d -j MASQUERADE
-A POSTROUTING -s 172.19.0.0/16 ! -o br-3608cd72cb8c -j MASQUERADE
-A POSTROUTING -s 10.0.0.0/24 -j SNAT --to-source 192.168.1.5
-A POSTROUTING -s 172.17.0.2/32 -d 172.17.0.2/32 -p tcp -m tcp --dport 6969 -j MASQUERADE
-A POSTROUTING -s 172.17.0.2/32 -d 172.17.0.2/32 -p udp -m udp --dport 6969 -j MASQUERADE
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A DOCKER -i docker0 -j RETURN
-A DOCKER -i br-aa7ab3e7773d -j RETURN
-A DOCKER -i br-3608cd72cb8c -j RETURN
-A DOCKER ! -i docker0 -p tcp -m tcp --dport 6969 -j DNAT --to-destination 172.17.0.2:6969
-A DOCKER ! -i docker0 -p udp -m udp --dport 6969 -j DNAT --to-destination 172.17.0.2:6969
COMMIT
# Completed on Sun Jan 10 00:31:59 2021
# Generated by xtables-save v1.8.2 on Sun Jan 10 00:31:59 2021
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:DOCKER - [0:0]
:DOCKER-ISOLATION-STAGE-1 - [0:0]
:DOCKER-ISOLATION-STAGE-2 - [0:0]
:DOCKER-USER - [0:0]
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-ISOLATION-STAGE-1
-A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o docker0 -j DOCKER
-A FORWARD -i docker0 ! -o docker0 -j ACCEPT
-A FORWARD -i docker0 -o docker0 -j ACCEPT
-A FORWARD -o br-aa7ab3e7773d -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o br-aa7ab3e7773d -j DOCKER
-A FORWARD -i br-aa7ab3e7773d ! -o br-aa7ab3e7773d -j ACCEPT
-A FORWARD -i br-aa7ab3e7773d -o br-aa7ab3e7773d -j ACCEPT
-A FORWARD -o br-3608cd72cb8c -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A FORWARD -o br-3608cd72cb8c -j DOCKER
-A FORWARD -i br-3608cd72cb8c ! -o br-3608cd72cb8c -j ACCEPT
-A FORWARD -i br-3608cd72cb8c -o br-3608cd72cb8c -j ACCEPT
-A DOCKER -d 172.17.0.2/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 6969 -j ACCEPT
-A DOCKER -d 172.17.0.2/32 ! -i docker0 -o docker0 -p udp -m udp --dport 6969 -j ACCEPT
-A DOCKER-ISOLATION-STAGE-1 -i docker0 ! -o docker0 -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i br-aa7ab3e7773d ! -o br-aa7ab3e7773d -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -i br-3608cd72cb8c ! -o br-3608cd72cb8c -j DOCKER-ISOLATION-STAGE-2
-A DOCKER-ISOLATION-STAGE-1 -j RETURN
-A DOCKER-ISOLATION-STAGE-2 -o docker0 -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o br-aa7ab3e7773d -j DROP
-A DOCKER-ISOLATION-STAGE-2 -o br-3608cd72cb8c -j DROP
-A DOCKER-ISOLATION-STAGE-2 -j RETURN
-A DOCKER-USER -j RETURN
COMMIT