New AMI doesn't have forward enabled

[justinsb]

The new AMI seems to be defaulting to -P FORWARD deny , at least with kopeio-networking (where I found it). Forwarding is needed for containers to talk to other nodes (for example)

[chrislovecnm]

Isn't this the Ubuntu / Debian default now?

[cgilmour]

Does the new AMI have docker 1.13+ or docker-ce 17.03+?
Those newer versions of Docker change the forward chain's policy to DROP.

[chrislovecnm]

https://github.com/kubernetes/kops/blob/master/nodeup/pkg/model/docker.go

Yes

Is it Docker or new iptables?

[chrislovecnm]

K8s 1.8 is using the new version of Docker to be clear.

[cgilmour]

Is it Docker or new iptables?

It's basically this part of docker: https://github.com/moby/moby/blob/v1.13.1/vendor/github.com/docker/libnetwork/drivers/bridge/setup_ip_forwarding.go#L42
Essentially, the setting for EnableIPForward defaults to true. When dockerd launches, it will check if it needs to change /proc/.../ip_forward to enable it. If it needs to enable that, it will also change iptables FORWARD policy to DROP.

Related issues: kubernetes/kubernetes#39823
Many of the CNI providers have already had to deal with this, by adding the necessary entries into the FORWARD chain.

[justinsb]

Should be fixed by #3977.