Exposing a node port of a service always uses the ingress network even if the service is on another network

[eddytruyen]

I have the opposite question of #33878

I have created a weave network and created a service of which the containers are attached to the weave network. The service also exposes a nodeport by means of the -p option of the docker service create command.

It would have expected that the nodeport is attached to the weave network. It isn't. It is attached to the ingress network. I cannot however access service ips from a the client that is running outside the swarm cluster, so I must use nodeports.

Doesn't this break the advantage of having specialized network plugins for specific network topologies and application-specific requirements?

[eddytruyen]

I understand from #33878 that the ingress network is never used for inter-container networking.

My application is a replicated mongodb cluster consisting of multiple mongodb containers each exposed as a separate service. All services are connected to a weave network with IP endpoints 10.0.0.2, 10.0.0.4, 10.0.0.6, etc, I configured the mongdb endpoints as x.x.x.x:30000 and x.x.x.x: 30001, x.x.x.x:30002 etc respectively such that an external mongodb client, that is running outside the swarm cluster, can correctly connect to all mongodb instances according to the mongodb's internal loadbalancing algorithm.

Now this mongodb application has been running without problems. However, the weave plugin has never been installed correctly on each node because I installed it before initiating the swarm. This fooled me in thinking that inter-container networking happens via the ingress network.

Is is correct to say that traffic hitting the 30000 nodeport is forwarded by the ingress-sbox container to the node on which the particular container runs. Then on that node it is passed on to the docker_gwbridge network to which the mongodb container is linked. So the weave network is never used for inter-container networking?

[thaJeztah]

The "ingress" networking forms the "routing mesh", which means that when you publish a service's port, the port will be accessible on any node in the swarm, even if no container for the service is running on that node. If you access the port on a node that does not have a container running for the service, the request is send over the ingress network to a node where a container is running.

If two services are connected to the same custom overlay network, they are able to communicate using the servicename as hostname. For example:

$ docker network create -d overlay mynetwork
$ docker service create --name backend --network=mynetwork nginx:alpine
$ docker service create --name frontend --network=mynetwork -p 80:80 nginx:alpine

From within a container for the frontend service, you can access http://backend:80 and connect to the backend service. This traffic goes through the mynetwork overlay network. Services that should only be accessible internally (i.e. accessed by other services, but not from outside the swarm) should not publish (-p / --publish) their ports; any port that a service listens on is accessible by other services/containers on the same custom network, without having to publish (or "expose") the port.

If you don't need the routing mesh (for example, you're using an external load-balancer that knows which hosts a service runs on), you can also use "host mode" publishing (--publish mode=host .....), which will publish the container's port on the host it's running on, but does not use the "ingress" networking to route traffic between nodes.

You can find more information in the documentation; https://docs.docker.com/engine/swarm/ingress/ and https://docs.docker.com/engine/userguide/networking/

Please keep in mind that the GitHub issue tracker is not intended as a general support forum,
but for reporting bugs and feature requests. For other type of questions, consider using one of;

• the Docker Support Forums - https://forums.docker.com
• the Docker community Slack channel (register here: http://dockr.ly/community)
• StackOverflow

I'm closing this issue because this is not a bug, but feel free to continue the conversation 👍