New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docker and ufw serious problems #4737

Closed
phlegx opened this Issue Mar 18, 2014 · 105 comments

Comments

Projects
None yet
@phlegx

phlegx commented Mar 18, 2014

Having installed ufw and blocking all incoming traffic by default (sudo ufw default deny) by running docker images that map the ports to my host machine, these mapped docker ports are accessible from outside, even though they are never allowed to be accessed.

Please note that on this machine DEFAULT_FORWARD_POLICY="ACCEPT" as described on this page http://docs.docker.io/en/latest/installation/ubuntulinux/#ufw has not been enabled and the property DEFAULT_FORWARD_POLICY="DROP" is still set.

Any ideas what might causing this?

Output of ufw status:

$ sudo ufw status verbose
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing)
New profiles: skip

To                         Action      From
--                         ------      ----
22                         ALLOW IN    Anywhere
443/tcp                    ALLOW IN    Anywhere
80/tcp                     ALLOW IN    Anywhere
5666                       ALLOW IN    95.xx.xx.xx
4949                       ALLOW IN    95.xx.xx.xx
22                         ALLOW IN    Anywhere (v6)
443/tcp                    ALLOW IN    Anywhere (v6)
80/tcp                     ALLOW IN    Anywhere (v6)

Here is the output of my rabbitmq via docker ps:

cf4028680530        188.xxx.xx.xx:5000/rabbitmq:latest           /bin/sh -c /usr/bin/   5 weeks ago         Up 5 days           0.0.0.0:15672->15672/tcp, 0.0.0.0:5672->5672/tcp   ecstatic_darwin/rabbitmq,focused_torvalds/rabbitmq,rabbitmq,sharp_bohr/rabbitmq,trusting_pike/rabbitm

Nmap test:

nmap -P0 example.com -p 15672

Starting Nmap 5.21 ( http://nmap.org ) at 2014-03-18 11:27 CET
Nmap scan report for example.com (188.xxx.xxx.xxx)
Host is up (0.048s latency).
PORT      STATE SERVICE
15672/tcp open  unknown

Nmap done: 1 IP address (1 host up) scanned in 0.09 seconds

General infos:

  • Ubuntu 12.04 server
$ uname -a
Linux production 3.8.0-29-generic #42~precise1-Ubuntu SMP Wed Aug 14 16:19:23 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux

$ docker version
Client version: 0.9.0
Go version (client): go1.2.1
Git commit (client): 2b3fdf2
Server version: 0.9.0
Git commit (server): 2b3fdf2
Go version (server): go1.2.1
Last stable version: 0.9.0

$ docker info
Containers: 12
Images: 315
Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Dirs: 339
WARNING: No swap limit support
@Soulou

This comment has been minimized.

Show comment
Hide comment
@Soulou

Soulou Mar 19, 2014

Contributor

Ufw is only setting things in the filter table. Basically, the docker traffic is diverted before and goes through the nat table, so ufw in this case is basically useless, if you want to drop the traffic for a container you need to add rules in the mangle/nat table.

http://cesarti.files.wordpress.com/2012/02/iptables.gif

Contributor

Soulou commented Mar 19, 2014

Ufw is only setting things in the filter table. Basically, the docker traffic is diverted before and goes through the nat table, so ufw in this case is basically useless, if you want to drop the traffic for a container you need to add rules in the mangle/nat table.

http://cesarti.files.wordpress.com/2012/02/iptables.gif

@honi

This comment has been minimized.

Show comment
Hide comment
@honi

honi Aug 7, 2014

@Soulou would you recommend adding to mangle or nat?

edited previous comment after some research

honi commented Aug 7, 2014

@Soulou would you recommend adding to mangle or nat?

edited previous comment after some research

@honi

This comment has been minimized.

Show comment
Hide comment
@honi

honi Aug 7, 2014

In my case I wanted to only allow a specific IP to connect to the exposed port. I've managed to do this with this rule.

It drops all connections to port <Port> if source IP is not <RemoteIP>. I suppose that if you would want to completely block all connections, then simply remove the ! -s <RemoteIP> bit.

iptables -I PREROUTING 1 -t mangle ! -s <RemoteIP> -p tcp --dport <Port> -j DROP

honi commented Aug 7, 2014

In my case I wanted to only allow a specific IP to connect to the exposed port. I've managed to do this with this rule.

It drops all connections to port <Port> if source IP is not <RemoteIP>. I suppose that if you would want to completely block all connections, then simply remove the ! -s <RemoteIP> bit.

iptables -I PREROUTING 1 -t mangle ! -s <RemoteIP> -p tcp --dport <Port> -j DROP
@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 Feb 20, 2015

Contributor

@honi In Docker 1.5 (maybe 1.4?) there were several iptables changes. Can you verify if this is still a problem with 1.5?

Contributor

cpuguy83 commented Feb 20, 2015

@honi In Docker 1.5 (maybe 1.4?) there were several iptables changes. Can you verify if this is still a problem with 1.5?

@saidimu

This comment has been minimized.

Show comment
Hide comment
@saidimu

saidimu May 19, 2015

@cpuguy83 I can confirm that this is still a problem with Docker 1.6

Adding --iptables=false to DOCKER_OPTS enables the expected behavior.

saidimu commented May 19, 2015

@cpuguy83 I can confirm that this is still a problem with Docker 1.6

Adding --iptables=false to DOCKER_OPTS enables the expected behavior.

@lauralorenz

This comment has been minimized.

Show comment
Hide comment
@lauralorenz

lauralorenz Jun 11, 2015

+1, still a problem with Docker 1.6

+1, still a problem with Docker 1.6

@cpuguy83

This comment has been minimized.

Show comment
Hide comment
Contributor

cpuguy83 commented Jun 11, 2015

@newhook

This comment has been minimized.

Show comment
Hide comment
@newhook

newhook Jul 8, 2015

So what is the story here? With Docker version 1.7.0, build 0baf609 on Ubuntu 14 this is still completely broken. Also the installaton instructions on https://docs.docker.com/installation/ubuntulinux/ have a section "Enable UFW forwarding" which appears to be unnecessary. Anyone installing docker on an Ubuntu box exposes any forwarded ports from their containers to the outside world, and even worse looking at the ufw rules gives no hints that this is occurring which is needless to stay pretty bad.

newhook commented Jul 8, 2015

So what is the story here? With Docker version 1.7.0, build 0baf609 on Ubuntu 14 this is still completely broken. Also the installaton instructions on https://docs.docker.com/installation/ubuntulinux/ have a section "Enable UFW forwarding" which appears to be unnecessary. Anyone installing docker on an Ubuntu box exposes any forwarded ports from their containers to the outside world, and even worse looking at the ufw rules gives no hints that this is occurring which is needless to stay pretty bad.

@VascoVisser

This comment has been minimized.

Show comment
Hide comment
@VascoVisser

VascoVisser Jul 14, 2015

Also with Docker 1.7 here. My experience is that Docker+UFW can facilitate two scenarios.

The first scenario and default behavior indeed exposes all mapped ports to the outside world; UFW cannot filter access to the containers.

Alternatively when setting the --iptables=false option, filtering incoming traffic with UFW works as expected. However, doing this stops the containers from making outbound connections to the outside world. Inter container communication still works. If you don't need outbound connectivity, then UFW together with --iptables=false seems to be a viable solution.

In my opinion a sensible default behavior for docker would be how it behaves currently with --iptables=false and allow outbound connections from the containers (or possibly make this easily configurable via a config option).

Also with Docker 1.7 here. My experience is that Docker+UFW can facilitate two scenarios.

The first scenario and default behavior indeed exposes all mapped ports to the outside world; UFW cannot filter access to the containers.

Alternatively when setting the --iptables=false option, filtering incoming traffic with UFW works as expected. However, doing this stops the containers from making outbound connections to the outside world. Inter container communication still works. If you don't need outbound connectivity, then UFW together with --iptables=false seems to be a viable solution.

In my opinion a sensible default behavior for docker would be how it behaves currently with --iptables=false and allow outbound connections from the containers (or possibly make this easily configurable via a config option).

@newhook

This comment has been minimized.

Show comment
Hide comment
@newhook

newhook Jul 14, 2015

I don't have a problem getting out. Did you try:

ufw allow in on docker0

newhook commented Jul 14, 2015

I don't have a problem getting out. Did you try:

ufw allow in on docker0

@VascoVisser

This comment has been minimized.

Show comment
Hide comment
@VascoVisser

VascoVisser Jul 14, 2015

@newhook ufw allow in on docker0 doesn't work for me. Even with ufw disabled I can't get out with --iptables=false.

@newhook ufw allow in on docker0 doesn't work for me. Even with ufw disabled I can't get out with --iptables=false.

@VascoVisser

This comment has been minimized.

Show comment
Hide comment
@VascoVisser

VascoVisser Jul 14, 2015

I have been experimenting with this a few hours now. I think I got it figured out.

... the installaton instructions on https://docs.docker.com/installation/ubuntulinux/ have a section "Enable UFW forwarding" which appears to be unnecessary.

The FORWARD chain does need policy set to ACCEPT if you have --iptables=false. It only appears this is not needed because the Docker installation package auto starts Docker and adds iptable rules the FORWARD chain. When afterwards you add --iptables=false to your config and restart docker those rules are still there. After the next reboot these rules will be gone and your containers wont be able to communicate unless you have the FORWARD chain policy set to ACCEPT.

What you need for a setup that allows filtering with UFW, inter container networking and outbound connectivity is

  • start docker with --iptables=false
  • FORWARD chain policy set to ACCEPT
  • add the following NAT rule:
    iptables -t nat -A POSTROUTING ! -o docker0 -s 172.17.0.0/16 -j MASQUERADE

I have been experimenting with this a few hours now. I think I got it figured out.

... the installaton instructions on https://docs.docker.com/installation/ubuntulinux/ have a section "Enable UFW forwarding" which appears to be unnecessary.

The FORWARD chain does need policy set to ACCEPT if you have --iptables=false. It only appears this is not needed because the Docker installation package auto starts Docker and adds iptable rules the FORWARD chain. When afterwards you add --iptables=false to your config and restart docker those rules are still there. After the next reboot these rules will be gone and your containers wont be able to communicate unless you have the FORWARD chain policy set to ACCEPT.

What you need for a setup that allows filtering with UFW, inter container networking and outbound connectivity is

  • start docker with --iptables=false
  • FORWARD chain policy set to ACCEPT
  • add the following NAT rule:
    iptables -t nat -A POSTROUTING ! -o docker0 -s 172.17.0.0/16 -j MASQUERADE
@newhook

This comment has been minimized.

Show comment
Hide comment
@newhook

newhook Jul 17, 2015

You are indeed correct! After a reboot communication is gone. Those rules seem to sort everything out. Thanks very much!

newhook commented Jul 17, 2015

You are indeed correct! After a reboot communication is gone. Those rules seem to sort everything out. Thanks very much!

@dakky

This comment has been minimized.

Show comment
Hide comment
@dakky

dakky Jul 20, 2015

start docker with --iptables=false
FORWARD chain policy set to ACCEPT
add the following NAT rule:
iptables -t nat -A POSTROUTING ! -o docker0 -s 172.17.0.0/16 -j MASQUERADE

with this setup its not possible anymore to access exposed ports from within a container:

  • Container 1: exposed port 12345
  • login to Container 2:
    telnet 172.17.42.1 12345 does not work anymore

dakky commented Jul 20, 2015

start docker with --iptables=false
FORWARD chain policy set to ACCEPT
add the following NAT rule:
iptables -t nat -A POSTROUTING ! -o docker0 -s 172.17.0.0/16 -j MASQUERADE

with this setup its not possible anymore to access exposed ports from within a container:

  • Container 1: exposed port 12345
  • login to Container 2:
    telnet 172.17.42.1 12345 does not work anymore
@VascoVisser

This comment has been minimized.

Show comment
Hide comment
@VascoVisser

VascoVisser Jul 20, 2015

@dakky I can't reproduce your issue. I have no issues with inter container communication. I suggest making sure you expose the port in your Dockerfile. Also try flushing your iptables rules and delete all user-defined chains before configuring and enabling UFW.

In any case it would be good if someone from the Docker team can verify that the configuration I propose makes sense.

@dakky I can't reproduce your issue. I have no issues with inter container communication. I suggest making sure you expose the port in your Dockerfile. Also try flushing your iptables rules and delete all user-defined chains before configuring and enabling UFW.

In any case it would be good if someone from the Docker team can verify that the configuration I propose makes sense.

@include

This comment has been minimized.

Show comment
Hide comment
@include

include Sep 19, 2015

Hi, any update on this? I can't find any official source how to fix this.
Currently I have a simple setup like:

/etc/defaults/ufw: DEFAULT_FORWARD_POLICY="ACCEPT"
/etc/defaults/docker: DOCKER_OPTS="--iptables=false"

ufw enable
ufw allow 22/tcp
ufw deny 80/tcp
ufw reload

host# docker run -it --rm -p 80:8000 ubuntu bash
container# apt-get update
container# python3 -m http.server

.1 I can reach Internet from container
.2 Internet can reach container via public-address:80

Am I missing something here? 10x

include commented Sep 19, 2015

Hi, any update on this? I can't find any official source how to fix this.
Currently I have a simple setup like:

/etc/defaults/ufw: DEFAULT_FORWARD_POLICY="ACCEPT"
/etc/defaults/docker: DOCKER_OPTS="--iptables=false"

ufw enable
ufw allow 22/tcp
ufw deny 80/tcp
ufw reload

host# docker run -it --rm -p 80:8000 ubuntu bash
container# apt-get update
container# python3 -m http.server

.1 I can reach Internet from container
.2 Internet can reach container via public-address:80

Am I missing something here? 10x

@teodor-pripoae

This comment has been minimized.

Show comment
Hide comment
@teodor-pripoae

teodor-pripoae Sep 30, 2015

I had managed to fix this through iptables mangle. The first 2 lines are optional if you want to allow access to some ports on eth1 (private network, if it exists).

sudo iptables -t mangle -A FORWARD -i eth1 -o docker0 -j ACCEPT
sudo iptables -t mangle -A FORWARD -i docker0 -o eth1 -j ACCEPT
sudo iptables -t mangle -A FORWARD -i docker0 -o eth0 -j ACCEPT
sudo iptables -t mangle -A FORWARD -i eth0 -o docker0 -j ACCEPT -m state --state ESTABLISHED,RELATED
sudo iptables -t mangle -A FORWARD -i eth0 -o docker0 -j DROP

I had managed to fix this through iptables mangle. The first 2 lines are optional if you want to allow access to some ports on eth1 (private network, if it exists).

sudo iptables -t mangle -A FORWARD -i eth1 -o docker0 -j ACCEPT
sudo iptables -t mangle -A FORWARD -i docker0 -o eth1 -j ACCEPT
sudo iptables -t mangle -A FORWARD -i docker0 -o eth0 -j ACCEPT
sudo iptables -t mangle -A FORWARD -i eth0 -o docker0 -j ACCEPT -m state --state ESTABLISHED,RELATED
sudo iptables -t mangle -A FORWARD -i eth0 -o docker0 -j DROP
@lenovouser

This comment has been minimized.

Show comment
Hide comment
@lenovouser

lenovouser Jan 6, 2016

This is still a problem. Is there a clear fix available? I don't expect a built-in solution. But maybe some iptables or or nat rules? I don't feel like testing all possible solutions in this issue now just to brick my system 😄

This is still a problem. Is there a clear fix available? I don't expect a built-in solution. But maybe some iptables or or nat rules? I don't feel like testing all possible solutions in this issue now just to brick my system 😄

@mikehaertl

This comment has been minimized.

Show comment
Hide comment
@mikehaertl

mikehaertl Mar 3, 2016

Guys, this is a serious security issue. Why is there no hint in the documentation for it? Only by accident I found out, that my MySQL Port is wide open to the world. I absolutely didn't expect that as I've used ufw before and it was reliable enough to not spend another thought on it. So I trusted the advice to change the forward policy to ACCEPT. I would never have expected that it basically completely suspends ufw.

Guys, this is a serious security issue. Why is there no hint in the documentation for it? Only by accident I found out, that my MySQL Port is wide open to the world. I absolutely didn't expect that as I've used ufw before and it was reliable enough to not spend another thought on it. So I trusted the advice to change the forward policy to ACCEPT. I would never have expected that it basically completely suspends ufw.

@mikehaertl

This comment has been minimized.

Show comment
Hide comment
@mikehaertl

mikehaertl Mar 3, 2016

For the record, the solution from @VascoVisser worked for me with docker V1.10. Here are the files I had to change:

  • Set DEFAULT_FORWARD_POLICY="ACCEPT" in /etc/default/ufw

  • Set DOCKER_OPTS="--iptables=false" in /etc/default/docker

  • Add the following block with my custom bridge's ip range to the top of /etc/ufw/before.rules:

    # nat Table rules
    *nat
    :POSTROUTING ACCEPT [0:0]
    
    # Forward traffic from eth1 through eth0.
    -A POSTROUTING -s 192.168.0.0/24 -o eth0 -j MASQUERADE
    
    # don't delete the 'COMMIT' line or these nat table rules won't be processed
    COMMIT
    

Note: I'm using a custom network for my docker containers, so you may have to change the 192.168.0.0 above to match your network range. The default is 172.17.0.0/16 as in Vasco's comment above.

UPDATE: On Ubuntu 16.04 things are different, because docker is started by systemd, so /etc/default/docker is ignored. The solution described here creates the file /etc/systemd/system/docker.service.d/noiptables.conf with this content

[Service]
ExecStart=
ExecStart=/usr/bin/docker daemon -H fd:// --iptables=false

and issue systemctl daemon-reload afterwards.

mikehaertl commented Mar 3, 2016

For the record, the solution from @VascoVisser worked for me with docker V1.10. Here are the files I had to change:

  • Set DEFAULT_FORWARD_POLICY="ACCEPT" in /etc/default/ufw

  • Set DOCKER_OPTS="--iptables=false" in /etc/default/docker

  • Add the following block with my custom bridge's ip range to the top of /etc/ufw/before.rules:

    # nat Table rules
    *nat
    :POSTROUTING ACCEPT [0:0]
    
    # Forward traffic from eth1 through eth0.
    -A POSTROUTING -s 192.168.0.0/24 -o eth0 -j MASQUERADE
    
    # don't delete the 'COMMIT' line or these nat table rules won't be processed
    COMMIT
    

Note: I'm using a custom network for my docker containers, so you may have to change the 192.168.0.0 above to match your network range. The default is 172.17.0.0/16 as in Vasco's comment above.

UPDATE: On Ubuntu 16.04 things are different, because docker is started by systemd, so /etc/default/docker is ignored. The solution described here creates the file /etc/systemd/system/docker.service.d/noiptables.conf with this content

[Service]
ExecStart=
ExecStart=/usr/bin/docker daemon -H fd:// --iptables=false

and issue systemctl daemon-reload afterwards.

@lenovouser

This comment has been minimized.

Show comment
Hide comment
@lenovouser

lenovouser Mar 3, 2016

@mikehaertl I want to mention that this is not really an issue just with UFW, as it is just another layer over iptables. This is a general problem in my opinion.

@mikehaertl I want to mention that this is not really an issue just with UFW, as it is just another layer over iptables. This is a general problem in my opinion.

@mikehaertl

This comment has been minimized.

Show comment
Hide comment
@mikehaertl

mikehaertl Mar 3, 2016

@lenovouser Thing is, that the documentation has some recommendation which sounds like "do this and everything is fine with ufw". But that's definitely not the case, so there should be big warning signs there.

@lenovouser Thing is, that the documentation has some recommendation which sounds like "do this and everything is fine with ufw". But that's definitely not the case, so there should be big warning signs there.

@lenovouser

This comment has been minimized.

Show comment
Hide comment
@lenovouser

lenovouser Mar 3, 2016

@mikehaertl Yep, definitely agree. I still don't really know how to use UFW with Docker properly. (After 5+ Months of going through / opening issues etc. Will try your solution later this day though 👍.

lenovouser commented Mar 3, 2016

@mikehaertl Yep, definitely agree. I still don't really know how to use UFW with Docker properly. (After 5+ Months of going through / opening issues etc. Will try your solution later this day though 👍.

@hbokh

This comment has been minimized.

Show comment
Hide comment
@hbokh

hbokh Apr 14, 2016

@mikehaertl Downside of your solution is that e.g. an nginx-container is only logging the docker0 address in access.log...

To get usable access-logging with real world IP addresses, I use jwilder/nginx-proxy in front of nginx with --net=host and X-Forwarded-For but this still feels like a slightly insecure workaround to me.

See this thread in the official Docker forums: Running multiple docker containers with UFW and --iptables=false”.

hbokh commented Apr 14, 2016

@mikehaertl Downside of your solution is that e.g. an nginx-container is only logging the docker0 address in access.log...

To get usable access-logging with real world IP addresses, I use jwilder/nginx-proxy in front of nginx with --net=host and X-Forwarded-For but this still feels like a slightly insecure workaround to me.

See this thread in the official Docker forums: Running multiple docker containers with UFW and --iptables=false”.

@mikehaertl

This comment has been minimized.

Show comment
Hide comment
@mikehaertl

mikehaertl Apr 15, 2016

@hbokh Are you sure that this is related to the change above? I can't see how it would modify the IP address. It only adds a MASQUERADE for connections initiated from a docker container. Anything else is untouched. So if a wrong IP address is logged, it should also happen without the modification.

@hbokh Are you sure that this is related to the change above? I can't see how it would modify the IP address. It only adds a MASQUERADE for connections initiated from a docker container. Anything else is untouched. So if a wrong IP address is logged, it should also happen without the modification.

@hbokh

This comment has been minimized.

Show comment
Hide comment
@hbokh

hbokh Apr 15, 2016

@mikehaertl It must be as far as I can see. I'm far from a ufw or iptables guy, so bear with me.
This is the access log when running nginx with the above 3 options enabled and ufw enabled - default deny, with some rules active to allow incoming traffic from specific addresses (nevermind the 500-error, since I skipped the fpm-container):

172.17.0.1 - - [15/Apr/2016:11:18:52 +0200] "GET / HTTP/1.1" 500 186 "-" "HTTPie/0.9.3"
172.17.0.1 - - [15/Apr/2016:11:18:54 +0200] "GET / HTTP/1.1" 500 186 "-" "HTTPie/0.9.3"

This is WITHOUT the three options and ufw completely disabled:

83.163.x.y - - [15/Apr/2016:11:26:18 +0200] "GET / HTTP/1.1" 500 186 "-" "HTTPie/0.9.3"
83.163.x.y - - [15/Apr/2016:11:26:19 +0200] "GET / HTTP/1.1" 500 186 "-" "HTTPie/0.9.3"

FYI Docker has added these lines to the top of the iptables chain in the last situation:

# Generated by iptables-save v1.4.21 on Fri Apr 15 11:33:02 2016
*nat
:PREROUTING ACCEPT [5:280]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A DOCKER -i docker0 -j RETURN
COMMIT
[ --- snip ---]

hbokh commented Apr 15, 2016

@mikehaertl It must be as far as I can see. I'm far from a ufw or iptables guy, so bear with me.
This is the access log when running nginx with the above 3 options enabled and ufw enabled - default deny, with some rules active to allow incoming traffic from specific addresses (nevermind the 500-error, since I skipped the fpm-container):

172.17.0.1 - - [15/Apr/2016:11:18:52 +0200] "GET / HTTP/1.1" 500 186 "-" "HTTPie/0.9.3"
172.17.0.1 - - [15/Apr/2016:11:18:54 +0200] "GET / HTTP/1.1" 500 186 "-" "HTTPie/0.9.3"

This is WITHOUT the three options and ufw completely disabled:

83.163.x.y - - [15/Apr/2016:11:26:18 +0200] "GET / HTTP/1.1" 500 186 "-" "HTTPie/0.9.3"
83.163.x.y - - [15/Apr/2016:11:26:19 +0200] "GET / HTTP/1.1" 500 186 "-" "HTTPie/0.9.3"

FYI Docker has added these lines to the top of the iptables chain in the last situation:

# Generated by iptables-save v1.4.21 on Fri Apr 15 11:33:02 2016
*nat
:PREROUTING ACCEPT [5:280]
:INPUT ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
:DOCKER - [0:0]
-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
-A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
-A DOCKER -i docker0 -j RETURN
COMMIT
[ --- snip ---]
@phlegx

This comment has been minimized.

Show comment
Hide comment
@phlegx

phlegx May 5, 2016

But what does then stop working with docker, if I disable the docker IP tables DOCKER_OPTS="--iptables=false", does anyone know about that?

I see no real solution out there currently. In my eyes docker should respect the manual IP tables that have been set (trough IP tables directly or trough ufw or else), however I think this is not doable.

So from my point of view currently it is just important that one keeps in mind, that once he opens a port with docker on 0.0.0.0 it will be available to the outside world, no matter what. If one decides not to open the port to the outside world trough docker, or if he runs an application on the host on that exact port without docker but natively, then all is fine and the firewall is still in place.

phlegx commented May 5, 2016

But what does then stop working with docker, if I disable the docker IP tables DOCKER_OPTS="--iptables=false", does anyone know about that?

I see no real solution out there currently. In my eyes docker should respect the manual IP tables that have been set (trough IP tables directly or trough ufw or else), however I think this is not doable.

So from my point of view currently it is just important that one keeps in mind, that once he opens a port with docker on 0.0.0.0 it will be available to the outside world, no matter what. If one decides not to open the port to the outside world trough docker, or if he runs an application on the host on that exact port without docker but natively, then all is fine and the firewall is still in place.

@mikehaertl

This comment has been minimized.

Show comment
Hide comment
@mikehaertl

mikehaertl May 18, 2016

@icecrime I don't think that this is limited to Version 0.9. It's also still happening in latest 1.11.

@icecrime I don't think that this is limited to Version 0.9. It's also still happening in latest 1.11.

@emq

This comment has been minimized.

Show comment
Hide comment
@emq

emq May 18, 2016

^ I can confirm that; actually stumbled upon this thread yesterday while I was setting up new server :p Vasco's solution does the job tho 👍

emq commented May 18, 2016

^ I can confirm that; actually stumbled upon this thread yesterday while I was setting up new server :p Vasco's solution does the job tho 👍

@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 May 18, 2016

Contributor

I'm inclined to say that there need only be a doc change.
Everything docker does is isolated to two chains ("DOCKER" and "DOCKER-ISOLATION") which docker itself creates.

Meanwhile, UFW thinks it is the single source of truth and does not actually read iptables for existing rules.

If you want to make sure ports aren't forwarded to an external interface then you need to make sure you specify which interface to bind to when creating the forwards, or you can even set the default interface on the daemon, which defaults to "0.0.0.0".

Alternatively, you can disable iptables support in docker all-together.

Contributor

cpuguy83 commented May 18, 2016

I'm inclined to say that there need only be a doc change.
Everything docker does is isolated to two chains ("DOCKER" and "DOCKER-ISOLATION") which docker itself creates.

Meanwhile, UFW thinks it is the single source of truth and does not actually read iptables for existing rules.

If you want to make sure ports aren't forwarded to an external interface then you need to make sure you specify which interface to bind to when creating the forwards, or you can even set the default interface on the daemon, which defaults to "0.0.0.0".

Alternatively, you can disable iptables support in docker all-together.

@heyman

This comment has been minimized.

Show comment
Hide comment
@heyman

heyman Sep 11, 2017

@alltheoptions If I've understood the issue correctly, it's caused by Docker inserting iptables rules before UFW's rules. I don't see how that can be fixed by UFW.

heyman commented Sep 11, 2017

@alltheoptions If I've understood the issue correctly, it's caused by Docker inserting iptables rules before UFW's rules. I don't see how that can be fixed by UFW.

@ksylvan

This comment has been minimized.

Show comment
Hide comment
@ksylvan

ksylvan Sep 14, 2017

I'd love to come up with a solution that does as @heyman suggests: Any way to use the new iptables DOCKER-USER table somehow to pass control over to ufw before jumping back into the docker-created rules?

ksylvan commented Sep 14, 2017

I'd love to come up with a solution that does as @heyman suggests: Any way to use the new iptables DOCKER-USER table somehow to pass control over to ufw before jumping back into the docker-created rules?

@ksylvan

This comment has been minimized.

Show comment
Hide comment
@ksylvan

ksylvan Sep 16, 2017

BTW just in case it wasn't obvious, this is still happening with the 17.06-ce version. Any new updates or clear solutions?

ksylvan commented Sep 16, 2017

BTW just in case it wasn't obvious, this is still happening with the 17.06-ce version. Any new updates or clear solutions?

@ksylvan

This comment has been minimized.

Show comment
Hide comment
@ksylvan

ksylvan Sep 16, 2017

FYI I have added this bug to the bug tracker for ufw: https://bugs.launchpad.net/ufw/+bug/1717648 - Maybe the ufw folks can come up with a fix from their side.

ksylvan commented Sep 16, 2017

FYI I have added this bug to the bug tracker for ufw: https://bugs.launchpad.net/ufw/+bug/1717648 - Maybe the ufw folks can come up with a fix from their side.

ksylvan added a commit to ksylvan/docker-mail-server that referenced this issue Sep 18, 2017

@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 Sep 18, 2017

Contributor

You would inject a jump rule into the DOCKER-USER chain that would jump to the appropriate ufw chain. I'm assuming ufw-user-input.
Untested, but something like:

iptables -I DOCKER-USER 1 -j ufw-user-input

This would insert a rule into DOCKER-USER at the first position that would jump to the ufw-user-input chain.

Contributor

cpuguy83 commented Sep 18, 2017

You would inject a jump rule into the DOCKER-USER chain that would jump to the appropriate ufw chain. I'm assuming ufw-user-input.
Untested, but something like:

iptables -I DOCKER-USER 1 -j ufw-user-input

This would insert a rule into DOCKER-USER at the first position that would jump to the ufw-user-input chain.

@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 Sep 18, 2017

Contributor

Maybe ufw-user-input is not perfect because it'll likely skip other ufw chains.

Contributor

cpuguy83 commented Sep 18, 2017

Maybe ufw-user-input is not perfect because it'll likely skip other ufw chains.

ericmccarthy7 added a commit to ericmccarthy7/jenkins-docker that referenced this issue Sep 29, 2017

@aequasi

This comment has been minimized.

Show comment
Hide comment
@aequasi

aequasi Oct 18, 2017

So, I've done what everyone has suggested, but it seems that when I use DEFAULT_FORWARD_POLICY="ACCEPT", all ports on the host are open, no matter what i do.

My rules:

root@baremetal1:~# ufw status verbose
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), allow (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
2375/tcp                       ALLOW IN    Anywhere
22/tcp                     ALLOW IN    Anywhere
9000                       DENY IN     Anywhere
9000 (v6)                  DENY IN     Anywhere (v6)

With our without the 9000 rules, i'm still able to hit a container published on port 9000 on the host

It actually appears like DNS resolutions end up failing if i set the default forward policy to DROP

TL;DR: I've added the masquerade settings, i've set the forward policy to accept, i've set the default incoming to deny, and outgoing to allow, enabled UFW, denied on 9000, but i can still hit a container on port 9000

aequasi commented Oct 18, 2017

So, I've done what everyone has suggested, but it seems that when I use DEFAULT_FORWARD_POLICY="ACCEPT", all ports on the host are open, no matter what i do.

My rules:

root@baremetal1:~# ufw status verbose
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), allow (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
2375/tcp                       ALLOW IN    Anywhere
22/tcp                     ALLOW IN    Anywhere
9000                       DENY IN     Anywhere
9000 (v6)                  DENY IN     Anywhere (v6)

With our without the 9000 rules, i'm still able to hit a container published on port 9000 on the host

It actually appears like DNS resolutions end up failing if i set the default forward policy to DROP

TL;DR: I've added the masquerade settings, i've set the forward policy to accept, i've set the default incoming to deny, and outgoing to allow, enabled UFW, denied on 9000, but i can still hit a container on port 9000

@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 Oct 18, 2017

Contributor

@aequasi You'll have to explain what exactly you did.

Contributor

cpuguy83 commented Oct 18, 2017

@aequasi You'll have to explain what exactly you did.

@mikehaertl

This comment has been minimized.

Show comment
Hide comment
@mikehaertl

mikehaertl Oct 18, 2017

@aequasi You didn't mention that you set --iptables=false. This is an important part because otherwhise docker will always override ufw rules. It could be necessary to reboot your machine to really get rid of all stale iptables rules.

@aequasi You didn't mention that you set --iptables=false. This is an important part because otherwhise docker will always override ufw rules. It could be necessary to reboot your machine to really get rid of all stale iptables rules.

@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 Oct 18, 2017

Contributor

otherwhise docker will always override ufw rules

Not if you put the rules in the DOCKER-USER chain.

Contributor

cpuguy83 commented Oct 18, 2017

otherwhise docker will always override ufw rules

Not if you put the rules in the DOCKER-USER chain.

@aequasi

This comment has been minimized.

Show comment
Hide comment
@aequasi

aequasi Oct 18, 2017

Sorry, forgot to mention that. I did add that to my dockerd args. The machine has been restarted a couple times.

I should mention that I'm running on Rancher, which may be doing stuff to the IPTables as well...?

aequasi commented Oct 18, 2017

Sorry, forgot to mention that. I did add that to my dockerd args. The machine has been restarted a couple times.

I should mention that I'm running on Rancher, which may be doing stuff to the IPTables as well...?

@aequasi

This comment has been minimized.

Show comment
Hide comment
@aequasi

aequasi Oct 18, 2017

After investigating some more, turns out Rancher is indeed adding stuff to iptables as well. Fairly annoying.

aequasi commented Oct 18, 2017

After investigating some more, turns out Rancher is indeed adding stuff to iptables as well. Fairly annoying.

@mikehaertl

This comment has been minimized.

Show comment
Hide comment
@mikehaertl

mikehaertl Oct 19, 2017

Not if you put the rules in the DOCKER-USER chain.

@cpuguy83 You don't have this option with ufw. It sets up its own set of chains which is kind of the main point. If you need to fiddle with chains manually again then ufw doesn't make much sense - but that's what this issue is all about (ufw and docker's iptables option and why they are no friends).

Not if you put the rules in the DOCKER-USER chain.

@cpuguy83 You don't have this option with ufw. It sets up its own set of chains which is kind of the main point. If you need to fiddle with chains manually again then ufw doesn't make much sense - but that's what this issue is all about (ufw and docker's iptables option and why they are no friends).

@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 Oct 19, 2017

Contributor

@mikehaertl I understand that, but as mentioned above you can make jump rules to the ufw chains.

Contributor

cpuguy83 commented Oct 19, 2017

@mikehaertl I understand that, but as mentioned above you can make jump rules to the ufw chains.

@mikehaertl

This comment has been minimized.

Show comment
Hide comment
@mikehaertl

mikehaertl Oct 19, 2017

Oh, I see now. Missed that. If I'm not completely missing the point you'd have to manually add those rules, right? So they could be put into /etc/ufw/before.rules to let ufw add them.

Oh, I see now. Missed that. If I'm not completely missing the point you'd have to manually add those rules, right? So they could be put into /etc/ufw/before.rules to let ufw add them.

@jgonsior

This comment has been minimized.

Show comment
Hide comment
@jgonsior

jgonsior Nov 7, 2017

I followed everything mentioned in this comment - now my docker ports are being secured by UFW as I want it. But am I the only one now, whose docker images are now unable to connect to the internet any longer, or did I missed a step mentioned somewhere else here?

jgonsior commented Nov 7, 2017

I followed everything mentioned in this comment - now my docker ports are being secured by UFW as I want it. But am I the only one now, whose docker images are now unable to connect to the internet any longer, or did I missed a step mentioned somewhere else here?

@hcguersoy

This comment has been minimized.

Show comment
Hide comment
@hcguersoy

hcguersoy Nov 8, 2017

@jgonsior The critical part is point 3. Did you've added the ! ?

Maybe some stuff has changed in Docker and / or Ubuntu, the original comment was based on Docker 1.13.1 and Ubuntu 16.04.

@jgonsior The critical part is point 3. Did you've added the ! ?

Maybe some stuff has changed in Docker and / or Ubuntu, the original comment was based on Docker 1.13.1 and Ubuntu 16.04.

@nurdism

This comment has been minimized.

Show comment
Hide comment
@nurdism

nurdism Nov 15, 2017

@jgonsior Running into same issue, worked before, I have a live setup with it working, using a fresh setup no longer working.

nurdism commented Nov 15, 2017

@jgonsior Running into same issue, worked before, I have a live setup with it working, using a fresh setup no longer working.

@nurdism

This comment has been minimized.

Show comment
Hide comment
@nurdism

nurdism Nov 15, 2017

ok found out ufw doesn't like where the rules are going.

I've changed the rules to /etc/ufw/before.rules put before the *filter rules

*nat
:POSTROUTING ACCEPT [0:0]
-A POSTROUTING ! -o docker0 -s 172.17.0.0/16 -j MASQUERADE
-A POSTROUTING ! -o docker_gwbridge -s 172.18.0.0/16 -j MASQUERADE
COMMIT

nurdism commented Nov 15, 2017

ok found out ufw doesn't like where the rules are going.

I've changed the rules to /etc/ufw/before.rules put before the *filter rules

*nat
:POSTROUTING ACCEPT [0:0]
-A POSTROUTING ! -o docker0 -s 172.17.0.0/16 -j MASQUERADE
-A POSTROUTING ! -o docker_gwbridge -s 172.18.0.0/16 -j MASQUERADE
COMMIT
@teodor-pripoae

This comment has been minimized.

Show comment
Hide comment
@teodor-pripoae

teodor-pripoae Jan 9, 2018

@hcguersoy Everything works ok for me but when I create a custom network, containers cannot connect to internet.

$ docker network create foo
$ docker run --network=foo -t -i alpine ping -c 1 github.com
ping: bad address 'github.com'

I simply cannot add each network to POSTROUTING table as above, since every network is created dynamically by our build pipeline.

@hcguersoy Everything works ok for me but when I create a custom network, containers cannot connect to internet.

$ docker network create foo
$ docker run --network=foo -t -i alpine ping -c 1 github.com
ping: bad address 'github.com'

I simply cannot add each network to POSTROUTING table as above, since every network is created dynamically by our build pipeline.

@hcguersoy

This comment has been minimized.

Show comment
Hide comment
@hcguersoy

hcguersoy Jan 10, 2018

@teodor-pripoae Have you excluded DNS problems, e.g. tried to ping on an IP address instead of name?
Spontaneous I've no idea beyond that.

@teodor-pripoae Have you excluded DNS problems, e.g. tried to ping on an IP address instead of name?
Spontaneous I've no idea beyond that.

@teodor-pripoae

This comment has been minimized.

Show comment
Hide comment
@teodor-pripoae

teodor-pripoae Jan 10, 2018

Yes, same problem.

$ docker run --network=foo -t -i alpine ping -c 1 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes

--- 8.8.8.8 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss

Yes, same problem.

$ docker run --network=foo -t -i alpine ping -c 1 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes

--- 8.8.8.8 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
@mikehaertl

This comment has been minimized.

Show comment
Hide comment
@mikehaertl

mikehaertl Jan 10, 2018

I simply cannot add each network to POSTROUTING table as above, since every network is created dynamically by our build pipeline.

Well, but this is crucial for your containers to connect to the internet. So unless you find a solution here, the problem will remain.

I simply cannot add each network to POSTROUTING table as above, since every network is created dynamically by our build pipeline.

Well, but this is crucial for your containers to connect to the internet. So unless you find a solution here, the problem will remain.

@teodor-pripoae

This comment has been minimized.

Show comment
Hide comment
@teodor-pripoae

teodor-pripoae Jan 10, 2018

@mikehaertl

I know but there should exist a solution for custom networks. I can't create a network, alter after.rules, restart ufw, run tests, alter after.rules again and then delete network.

@mikehaertl

I know but there should exist a solution for custom networks. I can't create a network, alter after.rules, restart ufw, run tests, alter after.rules again and then delete network.

@TheRealAstroboy

This comment has been minimized.

Show comment
Hide comment
@TheRealAstroboy

TheRealAstroboy Jan 12, 2018

@teodor-pripoae
I had the same problem until I kept only one custom network for all my dockers... now it works !

sudo docker network ls result :

NETWORK ID          NAME                  DRIVER              SCOPE
735783640c10        bridge                bridge              local
30e3a75a0136        elasticsearch_esnet   bridge              local
550c949a90cb        host                  host                local
86b8bb080f7f        none                  null                local

sudo docker ps result:

CONTAINER ID        IMAGE                                                 COMMAND                  CREATED             STATUS              PORTS                              NAMES
4d7b582e5ca6        analytics-lc                                            "analyid --pidfile="     3 hours ago         Up 43 minutes       0.0.0.0:6800->6800/tcp             analytics-lc
87736e84bef4        docker.elastic.co/kibana/kibana:6.1.1                 "/bin/bash /usr/loca…"   3 hours ago         Up 45 minutes       0.0.0.0:5601->5601/tcp             kibana
ef356b16e1fa        docker.elastic.co/elasticsearch/elasticsearch:6.1.1   "/usr/local/bin/dock…"   4 hours ago         Up 45 minutes       0.0.0.0:9200->9200/tcp, 9300/tcp   elasticsearch2

sudo iptables -L -n -t nat result :

Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination

Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination
MASQUERADE  all  --  10.8.0.0/24          0.0.0.0/0
MASQUERADE  all  --  172.17.0.0/16        0.0.0.0/0
MASQUERADE  all  --  172.18.0.0/16        0.0.0.0/0

sudo ufw status verbose numbered result :

Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), allow (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
22/tcp                     ALLOW IN    Anywhere
2000/udp                   ALLOW IN    Anywhere
9200                       ALLOW IN    10.8.0.0/16
5601                       ALLOW IN    10.8.0.0/16
2375/tcp                   ALLOW IN    10.8.0.0/16
Anywhere on docker0        ALLOW IN    Anywhere
22/tcp (v6)                ALLOW IN    Anywhere (v6)
2000/udp (v6)              ALLOW IN    Anywhere (v6)
Anywhere (v6) on docker0   ALLOW IN    Anywhere (v6)

Anywhere                   ALLOW OUT   Anywhere on tun0
Anywhere (v6)              ALLOW OUT   Anywhere (v6) on tun0

TheRealAstroboy commented Jan 12, 2018

@teodor-pripoae
I had the same problem until I kept only one custom network for all my dockers... now it works !

sudo docker network ls result :

NETWORK ID          NAME                  DRIVER              SCOPE
735783640c10        bridge                bridge              local
30e3a75a0136        elasticsearch_esnet   bridge              local
550c949a90cb        host                  host                local
86b8bb080f7f        none                  null                local

sudo docker ps result:

CONTAINER ID        IMAGE                                                 COMMAND                  CREATED             STATUS              PORTS                              NAMES
4d7b582e5ca6        analytics-lc                                            "analyid --pidfile="     3 hours ago         Up 43 minutes       0.0.0.0:6800->6800/tcp             analytics-lc
87736e84bef4        docker.elastic.co/kibana/kibana:6.1.1                 "/bin/bash /usr/loca…"   3 hours ago         Up 45 minutes       0.0.0.0:5601->5601/tcp             kibana
ef356b16e1fa        docker.elastic.co/elasticsearch/elasticsearch:6.1.1   "/usr/local/bin/dock…"   4 hours ago         Up 45 minutes       0.0.0.0:9200->9200/tcp, 9300/tcp   elasticsearch2

sudo iptables -L -n -t nat result :

Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination

Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination
MASQUERADE  all  --  10.8.0.0/24          0.0.0.0/0
MASQUERADE  all  --  172.17.0.0/16        0.0.0.0/0
MASQUERADE  all  --  172.18.0.0/16        0.0.0.0/0

sudo ufw status verbose numbered result :

Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), allow (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
22/tcp                     ALLOW IN    Anywhere
2000/udp                   ALLOW IN    Anywhere
9200                       ALLOW IN    10.8.0.0/16
5601                       ALLOW IN    10.8.0.0/16
2375/tcp                   ALLOW IN    10.8.0.0/16
Anywhere on docker0        ALLOW IN    Anywhere
22/tcp (v6)                ALLOW IN    Anywhere (v6)
2000/udp (v6)              ALLOW IN    Anywhere (v6)
Anywhere (v6) on docker0   ALLOW IN    Anywhere (v6)

Anywhere                   ALLOW OUT   Anywhere on tun0
Anywhere (v6)              ALLOW OUT   Anywhere (v6) on tun0
@teodor-pripoae

This comment has been minimized.

Show comment
Hide comment
@teodor-pripoae

teodor-pripoae Jan 14, 2018

I can't keep only one network, the build pipeline is creating a new one for every build and deletes it afterwards.

I fixed the problem by uninstalling ufw and using iptables.

I can't keep only one network, the build pipeline is creating a new one for every build and deletes it afterwards.

I fixed the problem by uninstalling ufw and using iptables.

@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 Jan 16, 2018

Contributor

Is this fully resolved by the use of the DOCKER-USER chain?

Contributor

cpuguy83 commented Jan 16, 2018

Is this fully resolved by the use of the DOCKER-USER chain?

@john-dev

This comment has been minimized.

Show comment
Hide comment
@john-dev

john-dev Apr 5, 2018

Any news on this?
My server is still open to the world..

john-dev commented Apr 5, 2018

Any news on this?
My server is still open to the world..

@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 Apr 5, 2018

Contributor

I'm going to close this. Docker includes support for a DOCKER-USER chain where which all traffic is configured to pass through. This is where rules can be added and won't be touched by Docker.

Thanks!

Contributor

cpuguy83 commented Apr 5, 2018

I'm going to close this. Docker includes support for a DOCKER-USER chain where which all traffic is configured to pass through. This is where rules can be added and won't be touched by Docker.

Thanks!

@cpuguy83 cpuguy83 closed this Apr 5, 2018

@tomholub

This comment has been minimized.

Show comment
Hide comment
@tomholub

tomholub May 26, 2018

@cpuguy83 Can you please elaborate in more detail about the recommended fix?

What I've see from you is:

iptables -I DOCKER-USER 1 -j ufw-user-input
  1. Should we use this command verbatim?
  2. Do we do this one time or on each host reboot?
  3. Is it safe to run this command repeatedly (eg during each deploy)? Or is it meant to be only run once? If once, how do we check if the required rule is already in place?
  4. After we run it, do we use ufw commands as usual (eg ufw allow 80/tcp) and it will work as expected?
  5. Is the command above the only change needed, or do we set any config files/other settings?
  6. Do we reload iptables somehow for this to take effect?

Thank you. Having clear guidance would be very helpful.

tomholub commented May 26, 2018

@cpuguy83 Can you please elaborate in more detail about the recommended fix?

What I've see from you is:

iptables -I DOCKER-USER 1 -j ufw-user-input
  1. Should we use this command verbatim?
  2. Do we do this one time or on each host reboot?
  3. Is it safe to run this command repeatedly (eg during each deploy)? Or is it meant to be only run once? If once, how do we check if the required rule is already in place?
  4. After we run it, do we use ufw commands as usual (eg ufw allow 80/tcp) and it will work as expected?
  5. Is the command above the only change needed, or do we set any config files/other settings?
  6. Do we reload iptables somehow for this to take effect?

Thank you. Having clear guidance would be very helpful.

@cpuguy83

This comment has been minimized.

Show comment
Hide comment
@cpuguy83

cpuguy83 May 26, 2018

Contributor
Contributor

cpuguy83 commented May 26, 2018

@tomholub

This comment has been minimized.

Show comment
Hide comment
@tomholub

tomholub May 26, 2018

Thanks for the speedy reply. I'm not sure that this solves it on my end:

# iptables -I DOCKER-USER 1 -j ufw-user-input

# iptables -S | grep ufw-user-input
-N ufw-user-input
-A DOCKER-USER -j ufw-user-input
-A ufw-before-input -j ufw-user-input
-A ufw-user-input -p tcp -m tcp --dport 22 -j ACCEPT
-A ufw-user-input -s ****censored****/32 -p tcp -m tcp --dport 26257 -j ACCEPT
-A ufw-user-input -s ****censored****/32 -p tcp -m tcp --dport 8080 -j ACCEPT
-A ufw-user-input -s ****censored****/32 -p tcp -m tcp --dport 26257 -j ACCEPT
-A ufw-user-input -s ****censored****/32 -p tcp -m tcp --dport 26257 -j ACCEPT
-A ufw-user-input -s ****censored****/32 -p tcp -m tcp --dport 26257 -j ACCEPT
-A ufw-user-input -s ****censored****/32 -p tcp -m tcp --dport 26257 -j ACCEPT
-A ufw-user-input -s ****censored****/32 -p tcp -m tcp --dport 8080 -j ACCEPT

# ufw status verbose
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), deny (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
22/tcp                     ALLOW IN    Anywhere                  
26257/tcp                  ALLOW IN    ****censored****           
8080/tcp                   ALLOW IN    ****censored****              
26257/tcp                  ALLOW IN    ****censored****              
26257/tcp                  ALLOW IN    ****censored****             
26257/tcp                  ALLOW IN    ****censored****             
26257/tcp                  ALLOW IN    ****censored****            
8080/tcp                   ALLOW IN    ****censored****            

Yet I can curl server-in-question:9911 from my local device, and get a response from a docker container that I expected to be blocked by the rules above.

tomholub commented May 26, 2018

Thanks for the speedy reply. I'm not sure that this solves it on my end:

# iptables -I DOCKER-USER 1 -j ufw-user-input

# iptables -S | grep ufw-user-input
-N ufw-user-input
-A DOCKER-USER -j ufw-user-input
-A ufw-before-input -j ufw-user-input
-A ufw-user-input -p tcp -m tcp --dport 22 -j ACCEPT
-A ufw-user-input -s ****censored****/32 -p tcp -m tcp --dport 26257 -j ACCEPT
-A ufw-user-input -s ****censored****/32 -p tcp -m tcp --dport 8080 -j ACCEPT
-A ufw-user-input -s ****censored****/32 -p tcp -m tcp --dport 26257 -j ACCEPT
-A ufw-user-input -s ****censored****/32 -p tcp -m tcp --dport 26257 -j ACCEPT
-A ufw-user-input -s ****censored****/32 -p tcp -m tcp --dport 26257 -j ACCEPT
-A ufw-user-input -s ****censored****/32 -p tcp -m tcp --dport 26257 -j ACCEPT
-A ufw-user-input -s ****censored****/32 -p tcp -m tcp --dport 8080 -j ACCEPT

# ufw status verbose
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), deny (routed)
New profiles: skip

To                         Action      From
--                         ------      ----
22/tcp                     ALLOW IN    Anywhere                  
26257/tcp                  ALLOW IN    ****censored****           
8080/tcp                   ALLOW IN    ****censored****              
26257/tcp                  ALLOW IN    ****censored****              
26257/tcp                  ALLOW IN    ****censored****             
26257/tcp                  ALLOW IN    ****censored****             
26257/tcp                  ALLOW IN    ****censored****            
8080/tcp                   ALLOW IN    ****censored****            

Yet I can curl server-in-question:9911 from my local device, and get a response from a docker container that I expected to be blocked by the rules above.

@tomholub

This comment has been minimized.

Show comment
Hide comment
@tomholub

tomholub May 27, 2018

I'm running docker-compose if that is relevant.

I'm running docker-compose if that is relevant.

@tomholub

This comment has been minimized.

Show comment
Hide comment
@tomholub

tomholub May 27, 2018

For the record, I was able to have a workable solution by:

  1. disabling iptables in docker: echo '{"iptables": false}' | sudo tee /etc/docker/daemon.json > /dev/null
  2. setting /etc/default/ufw back to default
  3. rebooting to flush all iptables rules
  4. deleting docker interfaces: docker network rm $(docker network ls | grep "bridge" | awk '/ / { print $1 }')
  5. binding docker containers directly to HOST network (network_mode: host in docker compose)
  6. setting ufw rules as usual

Incoming traffic is firewalled by my ufw rules, outgoing traffic works. Docker builds continue working when I add --network=host

This allows me to move on for the time being, but I would really like to have docker networking separated from my host network and still use ufw.

tomholub commented May 27, 2018

For the record, I was able to have a workable solution by:

  1. disabling iptables in docker: echo '{"iptables": false}' | sudo tee /etc/docker/daemon.json > /dev/null
  2. setting /etc/default/ufw back to default
  3. rebooting to flush all iptables rules
  4. deleting docker interfaces: docker network rm $(docker network ls | grep "bridge" | awk '/ / { print $1 }')
  5. binding docker containers directly to HOST network (network_mode: host in docker compose)
  6. setting ufw rules as usual

Incoming traffic is firewalled by my ufw rules, outgoing traffic works. Docker builds continue working when I add --network=host

This allows me to move on for the time being, but I would really like to have docker networking separated from my host network and still use ufw.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment