Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New docker networks cannot communicate to externally - Debian Buster #38974

Open
VariableDeclared opened this issue Mar 29, 2019 · 5 comments

Comments

@VariableDeclared
Copy link

commented Mar 29, 2019

When creating a user defined network I am unable to communicate with the outside world.

I believe this may be related to #38099.

The current work around I have for this is to run:

sudo update-alternatives --config iptables
**select iptables-legacy**
sudo service docker restart

Network can now communicate, this happens for all newly created docker networks, also continues to work if you run the above steps again to revert back to iptables-nft.

Steps to reproduce the issue:

  1. Upgrade to latest packages on Debian Buster (iptables v1.8.2)
  2. Upgrade to latest docker Docker version 18.09.4, build d14af54
  3. Create network using defaults: docker network create usr_net
  4. Attempt to ping google or any other external host

Describe the results you received:

I was unable to communicate with any host outside of the external interface, but was able to communicate with addresses on the machine's interface.

Describe the results you expected:

To be able to communicate with outside world, freely! :-P

Additional information you deem important (e.g. issue happens only occasionally):
Happens every time,

Tried following solutions BEFORE iptables solution:

  • sudo sysctl net.ipv4.conf.all.forwarding=1
  • sudo iptables -P FORWARD ACCEPT
  • sudo modprobe ip_conntrack_pptp
  • sudo modprobe ip_nat_pptp

Output of docker version:

Docker version 18.09.4, build d14af54

Output of docker info:

Containers: 1
 Running: 1
 Paused: 0
 Stopped: 0
Images: 37
Server Version: 18.09.4
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: bb71b10fd8f58240ca47fbb579b9d1028eea7c84
runc version: 2b18fe1d885ee5083ef9f0838fee39b62d653e30
init version: fec3683
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.19.0-2-amd64
Operating System: Debian GNU/Linux buster/sid
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 23.45GiB
Name: pete-debian-workstation-local
ID: DVCT:DAM3:2B5C:OTKL:UVXW:EDVP:YH5W:EV5V:XOTY:DXNI:TOEO:UIK5
Docker Root Dir: /home/pjds/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine

Additional environment details (AWS, VirtualBox, physical, etc.):
Physical

@thaJeztah

This comment has been minimized.

Copy link
Member

commented Apr 1, 2019

ping @arkodg PTAL

looks related to docker/libnetwork#2343 / docker/libnetwork#2344. Wondering if this is due to upgrade scenarios where the docker was installed before the revert (although iptables rules should be re-created when the docker service is restarted?)

@thaJeztah thaJeztah added this to backlog in maintainers-session Apr 1, 2019

@kolyshkin

This comment has been minimized.

Copy link
Contributor

commented Jun 20, 2019

So it looks like new iptables from Debian Buster is not fully compatible with the old/traditional iptables. Might be a problem with the distro; have you tried filing bug to Debian?

@kolyshkin

This comment has been minimized.

Copy link
Contributor

commented Jun 20, 2019

@seemethere maybe we can workaround it on a packaging level, requiring iptables-legacy package (rather than iptables) for debian buster?

@andrewhsu

This comment has been minimized.

Copy link
Contributor

commented Jun 20, 2019

Hmm...i see buster changed to nftables by default https://wiki.debian.org/nftables and discourages use of iptables.

Yes. Building new firewalls on top of iptables is discouraged.

@arkodg

This comment has been minimized.

Copy link
Contributor

commented Jun 20, 2019

There are two issues here

  1. libnetwork temporarily decided to use iptables-legacy which caused issues in networking in newer distros since different firewall managers were using different tools for packet filtering.
    This was fixed in docker/libnetwork#2343 and vendored back to Moby using #38983 so updating to the latest version of Docker CE should fix this issue
  2. After upgrading to the latest Docker CE, if you do see iptables errors in the dockerd logs with version 1.8.x, its due to issues with rule translations in iptables (that is now using a nft backend) that should be supported according to https://wiki.nftables.org/wiki-nftables/index.php/List_of_available_translations_via_iptables-translate_tool, users will need to downgrade/upgrade the iptables package to resolve this issue, or soft-link to the old stable iptables-legacy to circumvent this issue
    sudo update-alternatives --config iptables
    
    and choose iptables-legacy
@arkodg arkodg referenced this issue Jul 11, 2019
1 of 3 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
6 participants
You can’t perform that action at this time.