Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disabling the default bridge changes Docker's behavior regarding DOCKER-USER chain #47622

Open
anttikes opened this issue Mar 23, 2024 · 0 comments
Labels
kind/bug Bugs are bugs. The cause may or may not be known at triage time so debugging may be needed. status/0-triage

Comments

@anttikes
Copy link

anttikes commented Mar 23, 2024

Description

What I was trying to do is use "netfilter-persistent" together with Docker. The logic was that the rules governed by "netfilter-persistent" only touched INPUT and OUTPUT chains, and added the DOCKER-USER chain, placed it at the top of the FORWARD chain, and manipulated the rules in the DOCKER-USER chain. Docker would then continue from here, adding its own rules after the jump to DOCKER-USER.

The overall intent was to be able to control the forwarded traffic from Docker out to the Internet at large.

What I discovered was that adding "bridge": "none" to /etc/docker/daemon.json changes Docker's behavior in a rather unpredictable fashion.

By default, and when the setting is not there then Docker creates docker0 adapter, creates DOCKER-USER, adds jump to it to the top of the FORWARD chain before adding the other chains, isolation stages and all the other stuff. So, all is good. I can manually manipulate DOCKER-USER to enforce more stricter control.

When I add the setting, Docker's behavior changes. It no longer creates docker0 (which is expected) but it also does not create DOCKER-USER, and doesn't place a jump to it to the top of the FORWARD chain. Instead, it only creates the other chains and the isolation stages. This is rather perplexing, at least to me, and looks like a bug.

If I use iptables commands to add DOCKER-USER manually, add it to the top of FORWARD chain, and add rules to this chain, and put it to start before Docker (by manipulating "docker.service"'s systemd file) then when Docker starts it takes its own rules and inserts them before the jump to DOCKER-USER. This is also rather peculiar.

Note that I have a workaround for this, so all I want is for someone who knows Docker's internal behavior better than me take a look and explain why Docker omits the DOCKER-USER chain when the default bridge is set to "none".

Reproduce

  1. Install "moby-engine"
  2. Use docker network create --driver=bridge my-bridge to create a user-defined bridge network
  3. Use systemctl stop docker to shutdown Docker
  4. Use iptables -Fand iptables -X on "filter" and "nat" tables to ensure everything is clean
  5. Use systemctl start docker to restart Docker
  6. With iptables -S ensure that jump to DOCKER-USER chain is at the top of FORWARD chain
  7. Use systemctl stop docker to shutdown Docker
  8. Add "bridge": "none" to /etc/docker/daemon.json
  9. Use iptables -Fand iptables -X on "filter" and "nat" tables to clean all rules again
  10. Use systemctl start docker to restart Docker
  11. With iptables -S observe the jump to DOCKER-USER chain has not been added

Expected behavior

Expected behavior is that "bridge": "none" does not affect DOCKER-USER chain's addition or placement in the rules. Docker should behave consistently in this regard, and it should always ensure that jump to DOCKER-USER occurs before any other rules.

docker version

Client:
 Version:           24.0.9-1
 API version:       1.43
 Go version:        go1.20.13
 Git commit:        293681613032e6d1a39cc88115847d3984195c24
 Built:             Wed Jan 31 20:53:14 UTC 2024
 OS/Arch:           linux/amd64
 Context:           default

Server:
 Engine:
  Version:          24.0.9-1
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.13
  Git commit:       fca702de7f71362c8d103073c7e4a1d0a467fadd
  Built:            Thu Feb  1 00:12:23 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.30-1
  GitCommit:        d68034c2fe20ce0fc29692f87afced1edf7d77da
 runc:
  Version:          1.1.12-1
  GitCommit:        51d5e94601ceffbbd85688df1c928ecccbfa4685
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client:
 Version:    24.0.9-1
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  0.13.1-1
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx

Server:
 Containers: 1
  Running: 0
  Paused: 0
  Stopped: 1
 Images: 3
 Server Version: 24.0.9-1
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: journald
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: d68034c2fe20ce0fc29692f87afced1edf7d77da
 runc version: 51d5e94601ceffbbd85688df1c928ecccbfa4685
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
 Kernel Version: 5.15.146.1-microsoft-standard-WSL2
 Operating System: Ubuntu 22.04.4 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 12
 Total Memory: 15.58GiB
 Name: DESKTOP-5M6TJ8E
 ID: 9f4523df-2d8d-46b7-a5a1-069d47c921f4
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No blkio throttle.read_bps_device support
WARNING: No blkio throttle.write_bps_device support
WARNING: No blkio throttle.read_iops_device support
WARNING: No blkio throttle.write_iops_device support

Additional Info

This test was executed in Windows 11's WSL, using Ubuntu 22.04 distro.

@anttikes anttikes added kind/bug Bugs are bugs. The cause may or may not be known at triage time so debugging may be needed. status/0-triage labels Mar 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Bugs are bugs. The cause may or may not be known at triage time so debugging may be needed. status/0-triage
Projects
None yet
Development

No branches or pull requests

1 participant