Docker bypasses NixOS firewall exposing ports on the external interface

[pjones]

Using virtualisation.oci-containers.containers.<name>.ports exposes ports on the external interface regardless of firewall settings.

Docker injects its own firewall rules, bypassing the NixOS firewall.

To Reproduce

Configure a container and use the ports option:

{
  virtualisation.oci-containers.containers.foo = {
    # Other settings here ...
    ports = [ "8123:8123" ];
  };

  networking.firewall = {
    enable = true;
    allowedTCPPorts = lib.mkForce [ ];
    allowedUDPPorts = lib.mkForce [ ];
  };
}

Expected behavior

1. Port 8123 on the host will forward to the same port on the container (this works)
2. Port 8123 on the host blocked from external access via the firewall (Docker is bypassing this)

Additional context

My iptables skills are lacking, but I think this rule, created by Docker, is the problem:

  -A DOCKER -d 172.17.0.2/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 8123 -j ACCEPT

This was brought up in 2018 via #40507 but it looks like GitHub has deleted some comments and it's not clear why the issue was closed.

Notify maintainers

@adisbladis
@offlinehacker @tailhook @vdemeester @periklis

Metadata

• system: "x86_64-linux"
• host os: Linux 5.9.16, NixOS, 20.09pre-git (Nightingale)
• multi-user?: yes
• sandbox: yes
• version: nix-env (Nix) 2.3.10
• nixpkgs: /nix/store/3p4q8xc5y9kr57ysmd0s2s20s174pg23-a058d005b3cbb370bf171ebce01839dd6ff52222.tar.gz

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute:
# a list of nixos modules affected by the problem
module:
  - virtualisation.oci-containers.containers

[pjones]

@siers Could you shed some light on why #40507 was closed?

Thank you.

[adisbladis]

I recommend using the podman backend for oci-containers instead. This will honour firewall rules.

[pjones]

@adisbladis I agree that using podman is the preferred solution.

To resolve this particular bug either Docker needs to be removed as an oci-containers backend or if the ports option is used then Docker's firewall rules should be disabled and correctly crafted firewall rules inserted in their place.

[siers]

@pjones I think I thought it's going to be a "wontfix", but it doesn't warrant closing of the issue, in hindsight.

[arne-rusek]

I've worked around this using extraCommands in firewall config (docker-way to filter container traffic is to use DOCKER-USER chain):

iptables -N DOCKER-USER || true
iptables -F DOCKER-USER
iptables -A DOCKER-USER -i <external_iface> -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A DOCKER-USER -i <external_iface> -j DROP

This allows traffic originating from container to work but connectivity from outside is filtered.

I'm a total nixos newbie but having something similar by default when firewall is enabled and docker installed and adding rules based on some options etc would allow people to use docker safely?

[alarsyo]

Could someone add a severity:security label? This could be really bad if someone is unaware that they're exposing something externally

[wakira]

I had the same issue using the podman backend. Are you sure that podman works properly?

[andrevmatos]

This seems to be an issue with podman as well. I'm no expert in iptables, but it seems podman handles container's rules through FORWARD table, therefore INPUT rules like open ports are completely ignored. FORWARD default policy is ACCEPT, so everything gets accepted. podman also creates a few chains to handle this (CNI-FORWARD, CNI-ADMIN), so I managed to filter my container's input with something like this:

  networking = {
    firewall = {
      ...
      extraCommands = ''
        ip46tables -P FORWARD DROP
        ip46tables -N CNI-ADMIN || true
        ip46tables -A CNI-ADMIN -p tcp --dport 30303:30304 -j ACCEPT
        ip46tables -A CNI-ADMIN -p udp --dport 30303:30304 -j ACCEPT
      '';
      extraStopCommands = ''
        ip46tables -D CNI-ADMIN -p tcp --dport 30303:30304 -j ACCEPT || true
        ip46tables -D CNI-ADMIN -p udp --dport 30303:30304 -j ACCEPT || true
        ip46tables -P FORWARD ACCEPT
      '';
    };
  };

[computercam]

Just want to add that I believe you can disable this behavior by instructing the docker daemon not to manage iptables.

This can be done by passing the following arguments to dockerd:

      --iptables                              Enable addition of iptables rules (default true)
      --ip6tables                            Enable addition of ip6tables rules (default false)

In a configuration.nix file it can be set as follows:

virtualisation.docker.extraOptions = ''--iptables=false --ip6tables=false';

You can verify the additional flags have been passed via ssystemctl status docker | grep "libexec/docker/dockerd"

You may need to flush iptables rules to see this take effect, I believe that can be achieved by restarting. There may be another way to flush the rules without restarting but I'm not sure how to do this in NixOS.

More info at:

https://docs.docker.com/network/iptables/#prevent-docker-from-manipulating-iptables

[egasimus]

In the most common case, you don't even need to disable Docker's iptables integration or add custom config to your iptables - you can simpliy specify the port binding as "127.0.0.1:8123:8123" instead of "8123:8123". Then it will only listen on localhost.

[rokk4]

Just want to add that I believe you can disable this behavior by instructing the docker daemon not to manage iptables.

This can be done by passing the following arguments to dockerd:

      --iptables                              Enable addition of iptables rules (default true)
      --ip6tables                            Enable addition of ip6tables rules (default false)

In a configuration.nix file it can be set as follows:

virtualisation.docker.extraOptions = ''--iptables=false --ip6tables=false';

You can verify the additional flags have been passed via ssystemctl status docker | grep "libexec/docker/dockerd"

You may need to flush iptables rules to see this take effect, I believe that can be achieved by restarting. There may be another way to flush the rules without restarting but I'm not sure how to do this in NixOS.

More info at:

https://docs.docker.com/network/iptables/#prevent-docker-from-manipulating-iptables

No, this can fup container communcations, e.g. no internet/dns inside containers.
@egasimus solution is correct.

[rascal999]

I ran across this issue today - this is a problem which needs a solution, or at least a warning. The firewall is basically broken in certain situations, and if I didn't test access I would be unaware. https://nixos.wiki/wiki/Firewall does not mention that the firewall doesn't work with docker. I've added a warning about this.

@egasimus solution is valid but should not be required, the firewall should do its job. Has this issue really been open for ~18 months?

[cmoog]

Woah. This is a massive footgun. Maybe we should also add a warning message in the terminal if networking.firewall.enable && virtualisation.docker.enable. Should be pretty easy with this: https://github.com/NixOS/nixpkgs/blob/69a35ff92dc404bf04083be2fad4f3643b2152c9/nixos/doc/manual/development/assertions.section.md#warnings-sec-assertions-warnings

[asininemonkey]

Potential solution here. Perhaps a toggle to add said rules is required: https://github.com/chaifeng/ufw-docker

[aequitas]

I just stumbled on this with podman which also exhibits this behaviour which is as design and in line with Docker and CNI: containers/podman#15623

The rules for this reside in the NAT iptables table (see https://www.cni.dev/plugins/current/meta/portmap/#dnat) instead of the FILTER table where Docker does its thing.

The best solution seems to use explicit IP's in port mapping 127.0.0.1:8080:8080 or insert rules into the NAT chains.

[almino]

For me, it worked using 0.0.0.0:8080:8080. It also allowed me to use NGINX outside Docker on port 80.

[theAkito]

For me, it worked using 0.0.0.0:8080:8080. It also allowed me to use NGINX outside Docker on port 80.

0.0.0.0 means, everything is allowed, from anywhere.

The real Docker centric workaround for this is to use 127.0.0.1:8080:8080 as mentioned in the previous comment.

[RMTT]

Disable iptables in docker daemon setting is the simplest workaround i think.

[hacklschorsch]

https://docs.docker.com/network/packet-filtering-firewalls/ has a couple of different solutions for this.

Setting virtualisation.docker.daemon.settings = { ip = "127.0.0.1"; }; sets "the default bind address for containers" to localhost, making this less of a footgun. Unfortunately this "doesn't have any effect on Swarm services. Swarm services are always exposed on the 0.0.0.0 network interface." (and docker-compose behaves the same).

Since I regularly use docker-compose, I additionally blocked external traffic in the Docker chain.
(If the machine running Docker is also a router, you'll want to add rule(s) to not block legitimate traffic before the DROP rule - I leave these out here)

networking.firewall.extraCommands = ''
  iptables -N DOCKER-USER || true
  iptables -F DOCKER-USER
  iptables -A DOCKER-USER -i <external_iface> -m state --state RELATED,ESTABLISHED -j ACCEPT
  iptables -A DOCKER-USER -i <external-if> -j DROP
  iptables -A DOCKER-USER -j RETURN
'';