Containers can't communicate over macvlan internal network in swarm

[lemrouch]

Description

I'm trying to create macvlan private network for swarm service to communicate with non-docker system on the same VLAN.
In my case I have to create template networks for the macvlan one.
If I create the network as internal then containers can't communicate with each other.
If I create the network as non-internal it will change default gateway and break communication with rest of my network.

It's not possible to create internal config-only network.

Steps to reproduce the issue:

1. create config-only networks:
   on node1:
   docker network create --config-only --subnet 10.20.30.0/24 -o parent=enp1s0.30 --ip-range 10.20.30.192/27 private_net_template
   on node2:
   docker network create --config-only --subnet 10.20.30.0/24 -o parent=enp1s0.30 --ip-range 10.20.30.224/27 private_net_template

2. create network:
   docker network create -d macvlan --internal --scope swarm --attachable --config-from private_net_template private_net

Describe the results you received:

Internal parameter of the config-from network is overwritten by the config-only network.
Dummy interface is used as parent interface of the config-from private_net.

Describe the results you expected:

The private_net should use interface specified in it's config-from network.
Existing default gateway in container should not be changed for internal macvlan network.

Additional information you deem important (e.g. issue happens only occasionally):

I can create network without the --internal parameter but in such case it will change default gateway to non-existent ip from the network ip range which breaks communication.

Output of docker version:

Client:
 Version:           18.09.6
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        481bc77
 Built:             Sat May  4 02:36:00 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.6
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.8
  Git commit:       481bc77
  Built:            Sat May  4 01:59:36 2019
  OS/Arch:          linux/amd64
  Experimental:     false

Output of docker info:

Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 34
Server Version: 18.09.6
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: active
 NodeID: r2t1fkcjozyq3gm8vv7owwzxn
 Is Manager: true
 ClusterID: qynxz17vjb84n0vxbp2926s2o
 Managers: 1
 Nodes: 1
 Default Address Pool: 10.0.0.0/8  
 SubnetSize: 24
 Orchestration:
  Task History Retention Limit: 5
 Raft:
  Snapshot Interval: 10000
  Number of Old Snapshots to Retain: 0
  Heartbeat Tick: 1
  Election Tick: 10
 Dispatcher:
  Heartbeat Period: 5 seconds
 CA Configuration:
  Expiry Duration: 3 months
  Force Rotate: 0
 Autolock Managers: false
 Root Rotation In Progress: false
 Node Address: 10.64.10.57
 Manager Addresses:
  10.64.10.57:2377
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: bb71b10fd8f58240ca47fbb579b9d1028eea7c84
runc version: 2b18fe1d885ee5083ef9f0838fee39b62d653e30
init version: fec3683
Security Options:
 seccomp
  Profile: default
Kernel Version: 4.9.0-9-amd64
Operating System: Debian GNU/Linux 9 (stretch)
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 1.956GiB
Name: apex7
ID: J5R3:JKMK:N5JS:37VX:QH7K:QNBL:4HR3:XAGQ:EDU4:J3AA:RF6F:JMGF
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: 35
 Goroutines: 149
 System Time: 2019-06-21T10:45:10.561415698-04:00
 EventsListeners: 0
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine

WARNING: No swap limit support

Additional environment details (AWS, VirtualBox, physical, etc.):

This was tested on linux KVM VMs but this doesn't really matter.

[lemrouch]

Can somebody help me create "vendor bump" pull request?
I need several PRs to merge to fix this issue:

• Macvlan internal network should not change default gateway libnetwork#2407
• Macvlan network handles netlabel.Internal wrong libnetwork#2411
• Allow network with --config-from to be --internal libnetwork#2414
• Fix internal macvlan network to work in swarm libnetwork#2419

[thaJeztah]

opened #40596 to vendor the last missing PR from that list