Proper handling of NAT corner cases

[jpetazzo]

When allocating a port for a container (with -p 42), a NAT rule is inserted into the host's netfilter tables.

The initial implementation did insert a single rule in the PREROUTING table, which led to two issues:

• local traffic wasn't translated (i.e. connecting to the public port from an external machine would work, but connecting to the same port from the docker host wouldn't work),
• outbound traffic directed to an external service matching the public port would be translated as well.

The first issue is partially addressed by #98. However, it doesn't handle traffic directed to localhost.

We want to address the following problems. In the following examples, let's assume that with started a container with -p 42, and that it was mapped to port 42424.

1. From the docker host itself, telnet localhost 42424 doesn't work.
2. From any container on the docker host, telnet google.com 42424 will loop back to our container.

Both problems are hard.

I don't know if it is possible to solve the first one with netfilter. I thought that it would involve one OUTPUT rule (to DNAT the traffic) and one POSTROUTING rule (to SNAT the traffic), but the latter doesn't work (the rule is never matched by the traffic).
The workaround might be to setup an userland proxy, bound to 127.0.0.1:42424, redirecting connections to the container. Not very elegant, though.

To solve the second problem, we need fine-scoped rules. It would be convenient to know the outbound interface (i.e., if the traffic is outbound to eth0, do not DNAT it), but unfortunately, that cannot work, since the outbound interface is known after the PREROUTING chain is traversed (and this is the chain where we would insert the DNAT rule).

It looks like the only solution would be to enumerate all the local addresses of the host, and to maintain rules for all of them.

[dhrp]

+1 for making it possible to connect to localhost:port

With my vagrant install connecting to $(hostname) (as in the example) also doesn't work because it also resolves to 127.0.0.1

[unclejack]

I've looked into this and it won't be possible to use localhost at all without something like portfwd.

I'll provide a solution for the second problem (the one which causes ports to loop back to the container).

@jpetazzo You can assign this issue to me.

[unclejack]

Instead of trying to list all IPs of the host so that we allow the containers to connect on any of them, we could just pass to docker the interfaces we want it to take into account.

For example, I might have 6 interfaces and only care about allowing docker hosts to connect to each other via the IP of one single interface. So I'd pass docker -d some argument like -forward-if=tap0.
Docker would then get the IP of that interface and set up a rule to forward only traffic going to the IP of tap0.

Also, if I want to connect to any container, I have to use that IP + port, not localhost, nor any other IP of the host.

[jpetazzo]

Assigned to you, thanks!

Agreed on first issue (using localhost): it requires some userland proxy.

About second issue: "we could just pass to docker the interfaces we want it to take into account" — I think it won't be simpler than enumerating all of them, and some cases might even be tricky (e.g. what if I want to catch traffic directed to one specific secondary address or alias?)

IMHO, using all addresses would be "good enough" at first; and having a way to specify interfaces and/or IP addresses would be a nice bonus.

[unclejack]

@jpetazzo In that case, we could specify IPs on which we want docker to make the mapped ports available.
Would this be ok?

Also, it's easier to specify the IPs from which we map those ports from to the containers than enumerating all the interfaces and letting traffic through for all of them. It's also better to make things explicit than implicit in this case.

Specifying the IPs means we don't need to add code to get the list of interfaces.

So the flags would look something like: -map-ip=192.168.1.24 -map-ip=10.4.0.2.

For those who don't want to have to pass IPs to docker, we can also allow them to use the interface name. We won't have to obtain the IP or IPs of the interface(s) in that case either.

[jpetazzo]

In that case, we could specify IPs on which we want docker to make the mapped ports available.
Would this be ok?

Yes, I think that would be perfect.

we can also allow them to use the interface name.
We won't have to obtain the IP or IPs of the interface(s) in that case either.

I don't understand — if people specify an interface name, how are we going to build the iptables rules?
IIRC, we want to setup DNAT rules, and those rules have to be in the PREROUTING chain, where the output interface (-o flag) isn't available, right?

[unclejack]

This issue is done, so it can be closed.