New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No DOCKER chain in iptables #1871

Closed
brianclements opened this Issue Sep 12, 2013 · 42 comments

Comments

Projects
None yet
@brianclements

brianclements commented Sep 12, 2013

I've been doing this tutorial and had trouble at the step:

DJANGO_APP=$(docker run -d -i -t -p 8000:8000 ubuntu /bin/bash)
WARNING: Docker detected local DNS server on resolv.conf. Using default external servers: [8.8.8.8 8.8.4.4]
WARNING: IPv4 forwarding is disabled.
2013/09/12 12:52:02 Error: Error starting container 037aac8ca3ab: iptables failed: iptables -t nat -A DOCKER -p tcp --dport 8000 ! -i docker0 -j DNAT --to-destination 172.17.0.3:8000

So trying it manually gave me this:

sudo iptables -t nat -A DOCKER -p tcp --dport 8000 ! -i docker0 -j DNAT --to-destination 172.17.0.3:8000
iptables: No chain/target/match by that name.

This is verified by checking (note: I deleted the rules, just kept the chain names):

sudo iptables -L -n --line-numbers
Chain INPUT (policy DROP)
...

Chain FORWARD (policy DROP)
...

Chain OUTPUT (policy DROP)
...

Chain INBOUND (4 references)
...

Chain LOG_FILTER (5 references)
...

Chain LSI (2 references)
...

Chain LSO (0 references)
...

Chain OUTBOUND (3 references)
...

Any ideas? I did have an issues just before this where I couldn't access the internet at all from within containers, but "sharing" the internet locally with device docker0 using firestarter (gufw just doesn't work on my ubuntu 12.04) fixed it.

@crosbymichael

This comment has been minimized.

Show comment
Hide comment
@crosbymichael

crosbymichael Oct 4, 2013

Contributor

@Briguy2k

Can you look for the DOCKER chain using sudo iptables -L -n -t nat ??

Contributor

crosbymichael commented Oct 4, 2013

@Briguy2k

Can you look for the DOCKER chain using sudo iptables -L -n -t nat ??

@brianclements

This comment has been minimized.

Show comment
Hide comment
@brianclements

brianclements Oct 4, 2013

@crosbymichael

I didn't see anything helpful there:

$ sudo iptables -L -n -t nat 
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         
MASQUERADE  all  --  0.0.0.0/0            0.0.0.0/0

brianclements commented Oct 4, 2013

@crosbymichael

I didn't see anything helpful there:

$ sudo iptables -L -n -t nat 
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         
MASQUERADE  all  --  0.0.0.0/0            0.0.0.0/0
@matteosister

This comment has been minimized.

Show comment
Hide comment
@matteosister

matteosister Nov 9, 2013

same problem here....any solution?

matteosister commented Nov 9, 2013

same problem here....any solution?

@crosbymichael

This comment has been minimized.

Show comment
Hide comment
@crosbymichael

crosbymichael Nov 9, 2013

Contributor

@brianclements @matteosister

I'm guessing that you have the docker0 bridge so you can try to create the chain manually.

iptables -t nat -N DOCKER
iptables -t nat -A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
iptables -t nat -A PREROUTING -m addrtype --dst-type LOCAL ! --dst 127.0.0.0/8 -j DOCKER
Contributor

crosbymichael commented Nov 9, 2013

@brianclements @matteosister

I'm guessing that you have the docker0 bridge so you can try to create the chain manually.

iptables -t nat -N DOCKER
iptables -t nat -A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
iptables -t nat -A PREROUTING -m addrtype --dst-type LOCAL ! --dst 127.0.0.0/8 -j DOCKER
@matteosister

This comment has been minimized.

Show comment
Hide comment
@matteosister

matteosister Nov 10, 2013

@crosbymichael I've got the docker0 bridge, but the commands you gave didn't solve the issue....
Is there a way to restart from scratch?

matteosister commented Nov 10, 2013

@crosbymichael I've got the docker0 bridge, but the commands you gave didn't solve the issue....
Is there a way to restart from scratch?

@crosbymichael

This comment has been minimized.

Show comment
Hide comment
@crosbymichael

crosbymichael Nov 10, 2013

Contributor

You can remove the docker0 bridge and have docker recreate the network setup.

Michael Crosby

On Nov 10, 2013, at 10:00 AM, Matteo Giachino notifications@github.com wrote:

@crosbymichael I've got the docker0 bridge, but the commands you gave didn't solve the issue....
Is there a way to restart from scratch?


Reply to this email directly or view it on GitHub.

Contributor

crosbymichael commented Nov 10, 2013

You can remove the docker0 bridge and have docker recreate the network setup.

Michael Crosby

On Nov 10, 2013, at 10:00 AM, Matteo Giachino notifications@github.com wrote:

@crosbymichael I've got the docker0 bridge, but the commands you gave didn't solve the issue....
Is there a way to restart from scratch?


Reply to this email directly or view it on GitHub.

@brianclements

This comment has been minimized.

Show comment
Hide comment
@brianclements

brianclements Nov 14, 2013

@crosbymichael
Your suggestions worked for me and I was finally able to complete the tutorial in the original post exactly as directed, even down to the working django web server. However when I restarted, all the iptables changes were undone. Any tips on where I should start looking to make those permanent?

brianclements commented Nov 14, 2013

@crosbymichael
Your suggestions worked for me and I was finally able to complete the tutorial in the original post exactly as directed, even down to the working django web server. However when I restarted, all the iptables changes were undone. Any tips on where I should start looking to make those permanent?

@matteosister

This comment has been minimized.

Show comment
Hide comment
@matteosister

matteosister Nov 18, 2013

@crosbymichael for me nothing worked. Even after deleting the bridge still gives me the same error...

matteosister commented Nov 18, 2013

@crosbymichael for me nothing worked. Even after deleting the bridge still gives me the same error...

@crosbymichael

This comment has been minimized.

Show comment
Hide comment
@crosbymichael

crosbymichael Nov 30, 2013

Contributor

@matteosister What host OS are you running on?

Contributor

crosbymichael commented Nov 30, 2013

@matteosister What host OS are you running on?

@shykes

This comment has been minimized.

Show comment
Hide comment
@shykes

shykes Jan 6, 2014

Collaborator

Tagging as /system/networking

Collaborator

shykes commented Jan 6, 2014

Tagging as /system/networking

@jpoimboe

This comment has been minimized.

Show comment
Hide comment
@jpoimboe

jpoimboe Jan 24, 2014

Contributor

@brianclements @matteosister do you still see the issue with the latest version of docker?

Contributor

jpoimboe commented Jan 24, 2014

@brianclements @matteosister do you still see the issue with the latest version of docker?

@brianclements

This comment has been minimized.

Show comment
Hide comment
@brianclements

brianclements Jan 24, 2014

Thanks for following up @jpoimboe, I have since installed docker on many other machines and had 0 occurrence of this issue on those machines. And I'm pretty sure that I've had other miscellaneous firewall issues on the machine that this issue was occurring on, so I'm going to go ahead and take myself out of this equation and say that it's some issue with my firewall here on this machine and not docker.

Curious though myself @matteosister, what is your host system that you have the issue on and is it a fresh install or a system upgrade? My particular system has been under probably 3 years of Ubuntu upgrades and I think my iptables has too many different versions and types of firewalls all grabbing at it and undoing each others changes.

brianclements commented Jan 24, 2014

Thanks for following up @jpoimboe, I have since installed docker on many other machines and had 0 occurrence of this issue on those machines. And I'm pretty sure that I've had other miscellaneous firewall issues on the machine that this issue was occurring on, so I'm going to go ahead and take myself out of this equation and say that it's some issue with my firewall here on this machine and not docker.

Curious though myself @matteosister, what is your host system that you have the issue on and is it a fresh install or a system upgrade? My particular system has been under probably 3 years of Ubuntu upgrades and I think my iptables has too many different versions and types of firewalls all grabbing at it and undoing each others changes.

@eliasp

This comment has been minimized.

Show comment
Hide comment
@eliasp

eliasp Jan 24, 2014

Contributor

@brianclements It still fails here:

Jan 25 00:12:21 moria docker[5337]: Unable to allow incoming packets: iptables failed: iptables -I FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT: iptables: No chain/target/match by that name.
  • Distribution: Gentoo
  • Kernel: 3.13.0
  • Docker: 0.7.6
  • Go: 1.2

Running iptables -t nat -L shows:

Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         
MASQUERADE  all  --  172.17.0.0/16       !172.17.0.0/16  

There's no other firewall framework/daemon (Shorewall, firewalld, ufw, …) enabled/active at the time of testing this, so there's no chance this is a conflict with any of them.

Contributor

eliasp commented Jan 24, 2014

@brianclements It still fails here:

Jan 25 00:12:21 moria docker[5337]: Unable to allow incoming packets: iptables failed: iptables -I FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT: iptables: No chain/target/match by that name.
  • Distribution: Gentoo
  • Kernel: 3.13.0
  • Docker: 0.7.6
  • Go: 1.2

Running iptables -t nat -L shows:

Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination         

Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination         
MASQUERADE  all  --  172.17.0.0/16       !172.17.0.0/16  

There's no other firewall framework/daemon (Shorewall, firewalld, ufw, …) enabled/active at the time of testing this, so there's no chance this is a conflict with any of them.

@tianon

This comment has been minimized.

Show comment
Hide comment
@tianon

tianon Jan 24, 2014

Member

@eliasp I wonder if you just found another kernel flag our ebuild is missing. Do you have CONFIG_NETFILTER_XT_MATCH_CONNTRACK set in your kernel config, by any chance? :)

Member

tianon commented Jan 24, 2014

@eliasp I wonder if you just found another kernel flag our ebuild is missing. Do you have CONFIG_NETFILTER_XT_MATCH_CONNTRACK set in your kernel config, by any chance? :)

@eliasp

This comment has been minimized.

Show comment
Hide comment
@eliasp

eliasp Jan 25, 2014

Contributor

@tianon That's the culprit:

# CONFIG_NETFILTER_XT_MATCH_CONNTRACK is not set

Added it and it works now! So this should be also added to the ebuild.

Contributor

eliasp commented Jan 25, 2014

@tianon That's the culprit:

# CONFIG_NETFILTER_XT_MATCH_CONNTRACK is not set

Added it and it works now! So this should be also added to the ebuild.

@brianclements

This comment has been minimized.

Show comment
Hide comment
@brianclements

brianclements Jan 25, 2014

I would like to try this as well. Could you please give some instructions?
I don't quite follow what you did.

brianclements commented Jan 25, 2014

I would like to try this as well. Could you please give some instructions?
I don't quite follow what you did.

@eliasp

This comment has been minimized.

Show comment
Hide comment
@eliasp

eliasp Jan 25, 2014

Contributor

@brianclements Check, whether your Kernel has support for netfilter's connection tracking builtin.

A reasonably well built Kernel should expose its features through /proc/config.gz, so running

zgrep NETFILTER_XT_MATCH_CONNTRACK /proc/config.gz

should tell you about the support for this feature in the currently running Kernel.
Some distributions tend to use another not so deterministic approach and just place a plain config-files for each Kernel in /boot, so you'd have to do something like:

grep NETFILTER_XT_MATCH_CONNTRACK /boot/config-$(uname -r)

If the result of this check is is not set, this feature is not provided by your Kernel and you have to customize your Kernel to get support for it.

If the result of this check is =m, the feature is built as a module, so you'll have to load the module (it is loaded automatically on demand here, I don't know what would make your Ubuntu not to load it automatically) by running:

modprobe xt_conntrack

If the result of this check is =y the feature is already built into your Kernel and your problem has another reason than missing support for NETFILTER_XT_MATCH_CONNTRACK.

Contributor

eliasp commented Jan 25, 2014

@brianclements Check, whether your Kernel has support for netfilter's connection tracking builtin.

A reasonably well built Kernel should expose its features through /proc/config.gz, so running

zgrep NETFILTER_XT_MATCH_CONNTRACK /proc/config.gz

should tell you about the support for this feature in the currently running Kernel.
Some distributions tend to use another not so deterministic approach and just place a plain config-files for each Kernel in /boot, so you'd have to do something like:

grep NETFILTER_XT_MATCH_CONNTRACK /boot/config-$(uname -r)

If the result of this check is is not set, this feature is not provided by your Kernel and you have to customize your Kernel to get support for it.

If the result of this check is =m, the feature is built as a module, so you'll have to load the module (it is loaded automatically on demand here, I don't know what would make your Ubuntu not to load it automatically) by running:

modprobe xt_conntrack

If the result of this check is =y the feature is already built into your Kernel and your problem has another reason than missing support for NETFILTER_XT_MATCH_CONNTRACK.

@brianclements

This comment has been minimized.

Show comment
Hide comment
@brianclements

brianclements Feb 4, 2014

So the result of: grep NETFILTER_XT_MATCH_CONNTRACK /boot/config-$(uname -r) was =m, but trying to load it with modprobe or adding it to /etc/modules or even having Ubuntu's UFW load it by adding it to /etc/default/ufw at the bottom via this section # Extra connection tracking modules to load., never changed it to =y. So I just tried reseting my firewall, deleting all chains and rules, completely purged lxc-docker AND lxc from my system (I think the mistake I made before was not purging lxc, which didn't clear out the faulty bridge), made sure the bridge was deleted with brtcl show, then finally reinstalled lxc-docker and this has finally fixed it for me. The chain persists and port forwarding works as it should. Thanks all for the help.

brianclements commented Feb 4, 2014

So the result of: grep NETFILTER_XT_MATCH_CONNTRACK /boot/config-$(uname -r) was =m, but trying to load it with modprobe or adding it to /etc/modules or even having Ubuntu's UFW load it by adding it to /etc/default/ufw at the bottom via this section # Extra connection tracking modules to load., never changed it to =y. So I just tried reseting my firewall, deleting all chains and rules, completely purged lxc-docker AND lxc from my system (I think the mistake I made before was not purging lxc, which didn't clear out the faulty bridge), made sure the bridge was deleted with brtcl show, then finally reinstalled lxc-docker and this has finally fixed it for me. The chain persists and port forwarding works as it should. Thanks all for the help.

@jpoimboe

This comment has been minimized.

Show comment
Hide comment
@jpoimboe

jpoimboe Feb 4, 2014

Contributor

Ok, I think this issue can be closed now.

Contributor

jpoimboe commented Feb 4, 2014

Ok, I think this issue can be closed now.

@crosbymichael

This comment has been minimized.

Show comment
Hide comment
@crosbymichael

crosbymichael Feb 4, 2014

Contributor

Thanks for the update

Contributor

crosbymichael commented Feb 4, 2014

Thanks for the update

@fxtentacle

This comment has been minimized.

Show comment
Hide comment
@fxtentacle

fxtentacle Feb 5, 2014

I've experienced precisely this issue on a paravirtualized Ubuntu 12.04. The fix was to switch the kernel from my hoster's precompiled binary (which doesn't include the iptables addrtype module) to ubuntu's regular paravirtualized kernel.

fxtentacle commented Feb 5, 2014

I've experienced precisely this issue on a paravirtualized Ubuntu 12.04. The fix was to switch the kernel from my hoster's precompiled binary (which doesn't include the iptables addrtype module) to ubuntu's regular paravirtualized kernel.

@jpoimboe

This comment has been minimized.

Show comment
Hide comment
@jpoimboe

jpoimboe Feb 5, 2014

Contributor

Hm, this kind of seems to be a common issue. Maybe we should add a kernel check to ensure that all the needed iptables modules are available. Or at least a better iptables error message to tell the user they might be missing something.

Contributor

jpoimboe commented Feb 5, 2014

Hm, this kind of seems to be a common issue. Maybe we should add a kernel check to ensure that all the needed iptables modules are available. Or at least a better iptables error message to tell the user they might be missing something.

@MrManor

This comment has been minimized.

Show comment
Hide comment
@MrManor

MrManor Oct 1, 2014

Edit: I have now traced this problem down on my OpenSUSE box and I add this info here for others google'ing for a solution. I don’t know about other distributions.

On Suse the problem occurs when dhcp is used to acquirer ip addresses. Every time a dhcp lease is renewed - then the SUSEfirewall script is rerun. This destroys the ip rule setup by docker.

MrManor commented Oct 1, 2014

Edit: I have now traced this problem down on my OpenSUSE box and I add this info here for others google'ing for a solution. I don’t know about other distributions.

On Suse the problem occurs when dhcp is used to acquirer ip addresses. Every time a dhcp lease is renewed - then the SUSEfirewall script is rerun. This destroys the ip rule setup by docker.

@Pensu

This comment has been minimized.

Show comment
Hide comment
@Pensu

Pensu Mar 20, 2015

Contributor

@MrManor I am using static IP address. Still I am getting this issue. What solution did you use?

Contributor

Pensu commented Mar 20, 2015

@MrManor I am using static IP address. Still I am getting this issue. What solution did you use?

@MrManor

This comment has been minimized.

Show comment
Hide comment
@MrManor

MrManor Mar 20, 2015

@Pensu As I am only playing around whith this on my own network I moved the hole thing to another machine and disabled SuSEfirewall2 completely.

MrManor commented Mar 20, 2015

@Pensu As I am only playing around whith this on my own network I moved the hole thing to another machine and disabled SuSEfirewall2 completely.

@stieg

This comment has been minimized.

Show comment
Hide comment
@stieg

stieg May 15, 2015

I just hit this issue on CentOS 7. The root cause for me is that when firewalld gets restarted then the rules disappear from the iptables. Here you can see this effect in action:

# systemctl start docker.service
# iptables -L | grep DOCKER
DOCKER     all  --  anywhere             anywhere
Chain DOCKER (1 references)
# systemctl restart firewalld.service
# iptables -L | grep DOCKER
#

This was seen on the following docker package:

Name        : docker
Arch        : x86_64
Version     : 1.6.0
Release     : 11.0.1.el7.centos
Size        : 32 M
Repo        : installed
From repo   : extras

The package is the most recent one available from CentOS repos.

So this is most definitely a problem on RHEL 7 / CentOS 7 because any restarting of the firewalld service will cause this problem. Not saying this event will happen often, but services should be able to restart at any time without creating this kind of problem.

Proper fix is to ensure you have all kernel modules loaded and that the needed chains are present in the iptables rule set prior to running the iptables configuration rules.

stieg commented May 15, 2015

I just hit this issue on CentOS 7. The root cause for me is that when firewalld gets restarted then the rules disappear from the iptables. Here you can see this effect in action:

# systemctl start docker.service
# iptables -L | grep DOCKER
DOCKER     all  --  anywhere             anywhere
Chain DOCKER (1 references)
# systemctl restart firewalld.service
# iptables -L | grep DOCKER
#

This was seen on the following docker package:

Name        : docker
Arch        : x86_64
Version     : 1.6.0
Release     : 11.0.1.el7.centos
Size        : 32 M
Repo        : installed
From repo   : extras

The package is the most recent one available from CentOS repos.

So this is most definitely a problem on RHEL 7 / CentOS 7 because any restarting of the firewalld service will cause this problem. Not saying this event will happen often, but services should be able to restart at any time without creating this kind of problem.

Proper fix is to ensure you have all kernel modules loaded and that the needed chains are present in the iptables rule set prior to running the iptables configuration rules.

@shanye1987

This comment has been minimized.

Show comment
Hide comment
@shanye1987

shanye1987 Jul 22, 2015

@stieg then,What solution did you use?

shanye1987 commented Jul 22, 2015

@stieg then,What solution did you use?

@stieg

This comment has been minimized.

Show comment
Hide comment
@stieg

stieg Jul 22, 2015

@shanye1987 We killed firewalld and removed the package. Since our services are on AWS we just use their security groups anyways.

stieg commented Jul 22, 2015

@shanye1987 We killed firewalld and removed the package. Since our services are on AWS we just use their security groups anyways.

@SandeepKarnawat

This comment has been minimized.

Show comment
Hide comment
@SandeepKarnawat

SandeepKarnawat Oct 7, 2015

I faced this issue when my iptables were stopped. After starting iptables docker was able to create the container with exposed ports.

SandeepKarnawat commented Oct 7, 2015

I faced this issue when my iptables were stopped. After starting iptables docker was able to create the container with exposed ports.

@lukecampbell

This comment has been minimized.

Show comment
Hide comment
@lukecampbell

lukecampbell Nov 19, 2015

On a CentOS 7 VM, I was able to fix the issue by restarting docker through systemd which recreated the appropriate iptables rules.

sudo systemctl restart docker.service

This assumes of course that you have docker managed by systemd

lukecampbell commented Nov 19, 2015

On a CentOS 7 VM, I was able to fix the issue by restarting docker through systemd which recreated the appropriate iptables rules.

sudo systemctl restart docker.service

This assumes of course that you have docker managed by systemd

@jdejoode

This comment has been minimized.

Show comment
Hide comment
@jdejoode

jdejoode Nov 30, 2015

Or if you used

sudo update-rc.d docker defaults

to manage docker:

sudo service docker restart

jdejoode commented Nov 30, 2015

Or if you used

sudo update-rc.d docker defaults

to manage docker:

sudo service docker restart
@davclark

This comment has been minimized.

Show comment
Hide comment
@davclark

davclark Feb 9, 2016

Just to clarify - this is still broke but the issue is closed?

davclark commented Feb 9, 2016

Just to clarify - this is still broke but the issue is closed?

@drptbl

This comment has been minimized.

Show comment
Hide comment
@drptbl

drptbl Feb 9, 2016

Still broke. Happening for me all the time with "selenium/node-chrome-debug" image from here https://github.com/SeleniumHQ/docker-selenium.

drptbl commented Feb 9, 2016

Still broke. Happening for me all the time with "selenium/node-chrome-debug" image from here https://github.com/SeleniumHQ/docker-selenium.

@mayank941

This comment has been minimized.

Show comment
Hide comment
@mayank941

mayank941 Apr 7, 2016

Just stop all the running container then
systemctl restart docker.service
after that you can start your containers

mayank941 commented Apr 7, 2016

Just stop all the running container then
systemctl restart docker.service
after that you can start your containers

@JoseAlban

This comment has been minimized.

Show comment
Hide comment
@JoseAlban

JoseAlban Apr 7, 2016

I had the same issue after restarting firewalld on Centos7 to apply a new rule, and fixed it after @stieg and @lukecampbell 's great insights.

OS: Centos7
Kernel: 3.10.0-327.4.4.el7.x86_64
Docker: version 1.10.3, build 20f81dd
Problem: docker fails to publish ports due to no 'DOCKER' chain present in iptables
Root Cause: firewalld reload screws up iptable state: sudo firewall-cmd --reload

Is it a firewalld bug, or just expected behaviour?
Possible Solution: make firewalld aware of docker's iptable chain, so a firewalld restart behaves properly?

docker will recreate iptable's 'DOCKER' chain after a sudo systemctl restart docker, which can be confirmed via sudo iptables -L -n -t nat | grep DOCKER

JoseAlban commented Apr 7, 2016

I had the same issue after restarting firewalld on Centos7 to apply a new rule, and fixed it after @stieg and @lukecampbell 's great insights.

OS: Centos7
Kernel: 3.10.0-327.4.4.el7.x86_64
Docker: version 1.10.3, build 20f81dd
Problem: docker fails to publish ports due to no 'DOCKER' chain present in iptables
Root Cause: firewalld reload screws up iptable state: sudo firewall-cmd --reload

Is it a firewalld bug, or just expected behaviour?
Possible Solution: make firewalld aware of docker's iptable chain, so a firewalld restart behaves properly?

docker will recreate iptable's 'DOCKER' chain after a sudo systemctl restart docker, which can be confirmed via sudo iptables -L -n -t nat | grep DOCKER

@mayank941

This comment has been minimized.

Show comment
Hide comment
@mayank941

mayank941 Apr 7, 2016

thanx

Regards
Mayank Chauhan
+919718266167

On Thu, Apr 7, 2016 at 4:55 PM, Jose Alban notifications@github.com wrote:

I had the same issue after restarting firewalld on Centos7 to apply a new
rule, and fixed it after @stieg https://github.com/stieg and
@lukecampbell https://github.com/lukecampbell 's great insights.

OS: Centos7
Kernel: 3.10.0-327.4.4.el7.x86_64
Problem: docker fails to publish ports due to no 'DOCKER' chain present in
iptables
Root Cause: firewalld reload screws up iptable state (probably, since I've
done it but gathered no evidence beforehand)
Solution: fix firewalld bug, docker is actually helpfully recreating
iptable's 'DOCKER' chain after a sudo systemctl restart docker, which can
be confirmed via sudo iptables -L -n -t nat | grep DOCKER


You are receiving this because you commented.
Reply to this email directly or view it on GitHub
#1871 (comment)

mayank941 commented Apr 7, 2016

thanx

Regards
Mayank Chauhan
+919718266167

On Thu, Apr 7, 2016 at 4:55 PM, Jose Alban notifications@github.com wrote:

I had the same issue after restarting firewalld on Centos7 to apply a new
rule, and fixed it after @stieg https://github.com/stieg and
@lukecampbell https://github.com/lukecampbell 's great insights.

OS: Centos7
Kernel: 3.10.0-327.4.4.el7.x86_64
Problem: docker fails to publish ports due to no 'DOCKER' chain present in
iptables
Root Cause: firewalld reload screws up iptable state (probably, since I've
done it but gathered no evidence beforehand)
Solution: fix firewalld bug, docker is actually helpfully recreating
iptable's 'DOCKER' chain after a sudo systemctl restart docker, which can
be confirmed via sudo iptables -L -n -t nat | grep DOCKER


You are receiving this because you commented.
Reply to this email directly or view it on GitHub
#1871 (comment)

@TeraHz

This comment has been minimized.

Show comment
Hide comment
@TeraHz

TeraHz May 20, 2016

For CentOS 6 the fix is:

$ sudo service docker restart
$ sudo service iptables save

That will save the required chain in the iptables settings and when it reloads it will recreate what's needed.

TeraHz commented May 20, 2016

For CentOS 6 the fix is:

$ sudo service docker restart
$ sudo service iptables save

That will save the required chain in the iptables settings and when it reloads it will recreate what's needed.

@timothyparez

This comment has been minimized.

Show comment
Hide comment
@timothyparez

timothyparez Jun 1, 2016

I had the same problem with v1.11.1 on Ubuntu 14.04 solved it like this:

sudo service docker restart
iptables-save > /etc/iptables/rules.v4

This requires iptables-persistent to be installed:

sudo apt-get install iptables-persistent

timothyparez commented Jun 1, 2016

I had the same problem with v1.11.1 on Ubuntu 14.04 solved it like this:

sudo service docker restart
iptables-save > /etc/iptables/rules.v4

This requires iptables-persistent to be installed:

sudo apt-get install iptables-persistent
@denysvitali

This comment has been minimized.

Show comment
Hide comment
@denysvitali

denysvitali Jun 22, 2016

Why was @mayank941 answer "downvoted"?

Removing docker0 with ip link remove docker0 and restarting docker afterward with systemctl restart docker fixed my issues...

denysvitali commented Jun 22, 2016

Why was @mayank941 answer "downvoted"?

Removing docker0 with ip link remove docker0 and restarting docker afterward with systemctl restart docker fixed my issues...

@giaosudau

This comment has been minimized.

Show comment
Hide comment
@giaosudau

giaosudau commented Jul 21, 2016

@mayank941 It works.

@richardkeit

This comment has been minimized.

Show comment
Hide comment
@richardkeit

richardkeit Aug 10, 2016

I was getting the below when trying to expose ports on a container

"Error": "driver failed programming external connectivity on endpoint delete (7a2524372e20fe17adff94a3d327994c41219739afeed4304109cdd2c01954c9): iptables failed: iptables --wait -t filter -A DOCKER ! -i docker0 -o docker0 -p tcp -d 172.17.0.3 --dport 9200 -j ACCEPT: iptables: No chain/target/match by that name.\n (exit status 1)",

To fix this, modify the /etc/systemd/system/docker.service or /etc/systemd/system/docker.service.d/docker.conf file (depends what you're using) to include:
--iptables=false
then
systemctl daemon-reload
service docker restart

richardkeit commented Aug 10, 2016

I was getting the below when trying to expose ports on a container

"Error": "driver failed programming external connectivity on endpoint delete (7a2524372e20fe17adff94a3d327994c41219739afeed4304109cdd2c01954c9): iptables failed: iptables --wait -t filter -A DOCKER ! -i docker0 -o docker0 -p tcp -d 172.17.0.3 --dport 9200 -j ACCEPT: iptables: No chain/target/match by that name.\n (exit status 1)",

To fix this, modify the /etc/systemd/system/docker.service or /etc/systemd/system/docker.service.d/docker.conf file (depends what you're using) to include:
--iptables=false
then
systemctl daemon-reload
service docker restart

@haridsv

This comment has been minimized.

Show comment
Hide comment
@haridsv

haridsv Aug 25, 2016

@denysvitali I was able to get this working by running ip link delete docker0. To restart docker, /etc/init.d/docker restart gave me the message Docker is managed via upstart, try using service docker, so I ran sudo service docker restart but it failed with the message Unknown job: docker, so I just ended up killing the dockerd and docker-containerd processes and they got automatically restarted and the iptable entries were also recreated.

haridsv commented Aug 25, 2016

@denysvitali I was able to get this working by running ip link delete docker0. To restart docker, /etc/init.d/docker restart gave me the message Docker is managed via upstart, try using service docker, so I ran sudo service docker restart but it failed with the message Unknown job: docker, so I just ended up killing the dockerd and docker-containerd processes and they got automatically restarted and the iptable entries were also recreated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment