Join GitHub today
Swarm/mesh workaround: --publish mode=host,target=80,published=80,protocol=tcp #34161
Some folks (including me) seem to be having serious problems with the routing mesh/ingress network (for example, see #32195 and #34136). These are apparent, in stable releases 17.03.0-ce, 17.03.2-ce, and **17.06.0-ce"". I haven't tried the other intermediate edge releases.
I'm going to try to workaround this by manually deploying my ingress services to all nodes as a simple container. Unfortunately, containers don't support secrets, so this means I'll need to modify the containers to accept secrets as environment variables. I'll also need to manually roll out container updates.
It would be nice if the
This would simply expose the service port on the docker host machine and disable the fancy networking. Then we could use a combination of the
Mesh routing is a fantastic feature, but it really needs to be airtight. It's been a year+ since 1.12 introduced this and I'm not sure it's ever been stable.
This is already possible with
docker service create \ --publish mode=host,target=80,published=80,protocol=tcp \ --name=web \ --mode=global \ nginx:alpine
host-mode publishing publishes container's ports directly to the host, skipping the ingress network (thus the routing mesh)
Docs can be found here; https://docs.docker.com/engine/swarm/services/#publish-ports, but may be a bit hard to find, so I just opened docker/docker.github.io#3913 to request better docs (contributions welcome
changed the title
Swarm/mesh workaround: add option "service create --mesh=false : option
Jul 19, 2017
A follow-up note for future generations:
I was finally able to fix the original ingress network problem by reducing the ingress network MTU from it's default value of 1500 to 1492 bytes. This problem happens when the Docker nodes are hosted as virtual machines by hypervisors like Hyper-V and XenServer. I'm guessing that the Docker ingress network works like this:
I haven't actually confirmed the theory above, but changing MTU to 1492 fixed my problem.
Here's the script I used to recreate the ingress network. Note that I needed to a a bit of delay between removing and recreating it to give the Docker a chance to actually delete the thing. The subnet and gateway settings below match the Docker defaults.
For containers, the MTU is set to 1450; https://github.com/docker/libnetwork/blob/5d113d19f93b14b7b6a2b2d1f2fc1a2c1b6f9b0d/drivers/overlay/joinleave.go#L67-L70
// Set the container interface and its peer MTU to 1450 to allow // for 50 bytes vxlan encap (inner eth header(14) + outer IP(20) + // outer UDP(8) + vxlan header(8)) mtu := n.maxMTU()
For the ingressnetwork, no option is set explicitly;
Interesting. I just read this guys experience with this in August here:
...and I've been playing around with Docker's
The documentation for MTU here says: Set the containers network MTU. I inspected the ingress and a couple other networks I created and they all report MTU=1492. So I'm betting this might be the best way to configure the MTU because it will ensure that any networks created after swarm setup (e.g. via Docker Stacks) will also get the new MTU by default.