[networking] prevent users to use the ingress network

[rogaha]

Assuming that the ingress network is only meant to be used internally, we should either:

1. Warm the users that they should not use that network to create services and block them.
2. Hide the ingress network from the docker network ls list as they are not able to manage it anyway.

I've seem lots of users using that network to create services.

@mavenugo @mrjana do you have any thoughts on that?

[thaJeztah]

ping @mrjana any ideas on this one?

[rogaha]

ping @mrjana @mavenugo

[mavenugo]

@rogaha yes. I agree. We should probably prevent users to use this network to attach services.

[Richard-Mathie]

@mavenugo arrrrgghh nooooo

Can this be configurable as it is important for testing and fault finding to be able to check the network connection to services. By all means It would be great to lock things down in production. But in development, and with docker's quit frankly flaky networking we need to be able to dig down and find the fault and this may involve using dig, nslookup and curl on the service. The ingress network is an overlay network, don't hide is as 'magic'.

btw, having a really hard time mapping a kafka services to use docker swarm mode. All i really want is to use docker swarm mode to simplify service orchestration and service discovery. But there are real issues with the dns not pointing, or forgetting about, or remembering dead services, and traffic not being transferred through the VIP and ingress endpoint. Without being able to see what is under the hood, and test the service at every level, fault finding and fixing this mess would be impossible.

[Richard-Mathie]

Also if you are serious about this and applying granular segmentation on services. A service (A) with a published port (and thus on the ingress network), shouldn't be able to connect to another service (B) on the ingress network on any port not published by (B). Or should that be A shouldn't be able to talk to B at all? The latter is a bit silly as A can talk to B through local host on the published port (Though this will go through the VIP as I understand).

[thaJeztah]

@Richard-Mathie could you open a separate issue for the problems you're having with kafka / networking?

[Richard-Mathie]

@thaJeztah I have done this now at #26594

[mavenugo]

@rogaha @thaJeztah @Richard-Mathie PTAL #27147. Based on the analysis and discussions around #24637, we think it is better to make ingress network as special one and not let user manipulate it. Pls comment asap.

[rogaha]

Agreed! Thanks for the heads up @mavenugo

[thaJeztah]

Looks like this was fixed in 1.12.2;

docker service create --network=ingress nginx:alpine
Error response from daemon: rpc error: code = 3 desc = Service cannot be explicitly attached to "ingress" network which is a swarm internal network