Added macvlan and ipvlan drivers #964

nerdalert · 2016-02-22T23:26:22Z

Notes at: https://gist.github.com/nerdalert/c0363c15d20986633fda

Signed-off-by: Brent Salisbury brent@docker.com

aboch · 2016-02-23T01:12:29Z

drivers/ipvlan/ipvlan_test.go

+import (
+	"testing"
+
+	"github.com/docker/libnetwork/driverapi"


You need to add this import for the CI to pass:
_ "github.com/docker/libnetwork/testutils"

Thanks fellas! @aboch @mavenugo that fixed it.

mavenugo · 2016-02-23T01:29:01Z

@nerdalert 👏 thanks for the contribution. We will review it and provide feedback.

aboch · 2016-02-23T01:49:45Z

drivers/ipvlan/ipvlan_state.go

+	d.Lock()
+	networks := d.networks
+	d.Unlock()
+	n, ok := networks[nid]


We don't need networks, we only need to lock around this line only,

d.Lock() n, ok := d.networks[nid] d.Unlock()

aboch · 2016-02-23T19:07:19Z

drivers/ipvlan/ipvlan_network.go

+	for label, value := range labels {
+		switch label {
+		case hostIfaceOpt:
+			// parse driver option '-o --ipvlan_mode'


I think this comment and the one below should be swapped.

- Notes at: https://gist.github.com/nerdalert/c0363c15d20986633fda Signed-off-by: Brent Salisbury <brent@docker.com>

mavenugo · 2016-03-02T12:03:22Z

drivers/macvlan/macvlan_setup.go

+			return nil
+		}
+	}
+	return fmt.Errorf("required kernel module '%s' was not found in /proc/modules, kernel version >= 3.19 is recommended", macvlanType)


Is this true ?
I thought macvlan is supported in earlier kernels ?

mavenugo · 2016-03-02T13:37:32Z

@nerdalert Given the importance of getting feedback from users, shall we get these drivers under experimental for 1.11 and get enough feedback & also fix the L3 routing use-case. WDYT ?

aboch · 2016-03-02T23:41:02Z

Agree with @mavenugo. Being two new drivers, it will take some time with incremental changes to get them where we want them to be.
Introducing them in experimental seems the right approach to me as well.

mavenugo · 2016-03-03T01:39:03Z

@nerdalert I understand the need for -o host_iface option in order for the macvlan/ipvlan to plumb to underlay.

$ sudo docker network create -d macvlan test
ERRO[0008] Handler for POST /v1.22/networks/create returned error: macvlan requires an interface from the docker host to be specified (usage: -o host_iface=eth)
Error response from daemon: macvlan requires an interface from the docker host to be specified (usage: -o host_iface=eth)

But, as discussed,in the above case, if the user has not specified the host interface, we should consider it as an --internal case and automatically create a dummy interface and let the network creation successful. WDYT ?

mavenugo · 2016-03-03T02:23:56Z

@nerdalert few more comments as discussed. (commenting it here for better tracking)

Supporting --internal by using dummy interface. We can infact implement --internal and a configuration without -o host_iface similarly with the dummy interface approach.
Handle --mac-address option appropriately. It should error out in ipvlan but macvlan driver must accept and honor it.
Port-Mapping is infact a tricky scenario for these drivers. I would suggest a warning message and ignoring the option is better than Error'ing out (considering the portability concerns.
When multiple networks try to share the same parent interface, we should error it out with just the Netlink's device is busy message. But if the message doesnt contain the parent interface info, then it is better to customize it by indicating that the user is trying to reuse the parent interface.

nerdalert · 2016-03-07T08:23:15Z

@mavenugo great suggestions, experimental sounds perfect.

--internal is now supported. If either the --internal flag is set or the --parent= interface is not included then a netlink dummy interface is setup to act as a master parent to attach a macvlan/ipvlan space device to. Once containers are brought up on that network they have full communications with one another but no where outside of the dummy parent interface.
Changed port-mapping back to a warning for compose backward compatibility as you mentioned.
Changed -o host_iface= to -o parent=.
One outstanding is setting the MAC addr on Macvlan mode. I will open an issue to track it. Netlink can set a user specified MAC since it is intended to be unique for each interface while Ipvlan uses the same MAC as the parent by design and tracks IP --> port mappings instead of MAC --> port mappings in it's lookup tables.
Added 50+ networking scenarios for vetting bugs in this script ipvlan-macvlan-it.sh that has example usage and capabilities.

mavenugo · 2016-03-07T14:01:55Z

drivers/ipvlan/ipvlan_network.go

+	}
+	if !parentExists(config.Parent) {
+		// if the parent iface was not specified, create a dummy link
+		if config.Parent == stringid.TruncateID(config.ID) {


I think it is better to check for config.Internal here.

mavenugo · 2016-03-07T14:30:04Z

@nerdalert tried your patch e2e & am happy to see tagged use-case working across multi-host with appropriate configuration (in my OSX & virtual-box bridged adapter). These 2 drivers are really going to make the underlay integration extremely simple and I cant wait to get user feedback on these.

Apart from a few minor comments above, this PR gets a big 👍 from me.

mavenugo · 2016-03-07T16:56:53Z

@nerdalert I have a couple of macvlan networks with and without parent interfaces. When I tried to create and use an ipvlan network, I get the following error. Which seems wrong.

$ sudo docker network create -d ipvlan ip300
51ceb9c3f7877e674da65861eb8a8e8770fe6a36a66d705c4f51d6a4c51b6a98

$ sudo docker run --net=ip300 -it busybox sh
WARN[9635] failed to cleanup ipc mounts:
failed to umount /var/lib/docker/containers/53dece5c8166d90457f6270fe9018b37ccd1f9c5567cfa09f45005d9734a2965/shm: no such file or directory
ERRO[9635] Handler for POST /v1.22/containers/53dece5c8166d90457f6270fe9018b37ccd1f9c5567cfa09f45005d9734a2965/start returned error: Failed creating veth2db7a06, ensure there are no macvlan networks using the same `-o parent`: operation not supported
docker: Error response from daemon: Failed creating veth2db7a06, ensure there are no macvlan networks using the same `-o parent`: operation not supported.

Also the same issue when I tried to create an entirely new parent interface

$ sudo docker network create -d ipvlan -o parent=eth3.345 ip345
df831819e495cd0d561bc6cd1f84afdd887acdbced4c2bfdabffcbbbe02cbbcf

$ sudo docker run --net=ip345 -it busybox sh
WARN[9809] failed to cleanup ipc mounts:
failed to umount /var/lib/docker/containers/0064e630b9f4df3902c04796ba23f63d7b0b1aa60567082e092caf09512f477c/shm: no such file or directory
ERRO[9809] Handler for POST /v1.22/containers/0064e630b9f4df3902c04796ba23f63d7b0b1aa60567082e092caf09512f477c/start returned error: Failed creating vethb59a154, ensure there are no macvlan networks using the same `-o parent`: operation not supported
docker: Error response from daemon: Failed creating vethb59a154, ensure there are no macvlan networks using the same `-o parent`: operation not supported.

Can you pls take a look @ it ?

fredhsu · 2016-03-07T23:43:12Z

I've been running some tests with an Arista ToR acting as the gateway for macvlan/ipvlan, with and without 802.1q. So far so good!

macvlan bridge mode:
https://gist.github.com/fredhsu/63344b94fc18c87f8ed2

ipvlan L2 mode with 802.1q:
https://gist.github.com/fredhsu/38d75e7651dae8506ba2

Let me know if there are any specific tests you would like to have run against physical hardware.

nerdalert · 2016-03-08T01:03:04Z

Thanks @fredhsu appreciate you testing the physical side of the equation.

nerdalert · 2016-03-08T05:03:47Z

Regarding the operation not supported. was kernel versions < 3.19. Rather then warning in driver init(), it now fails network creates for macvlan < 3.09 and ipvlan < 3.19.

mavenugo · 2016-03-08T13:56:20Z

Thanks @nerdalert.

LGTM

aboch · 2016-03-09T17:37:31Z

LGTM

ping @chenchun

Signed-off-by: Madhu Venugopal <madhu@docker.com>

chenchun · 2016-03-10T02:03:58Z

👍 Introducing them in experimental.

LGTM, thanks for your work @nerdalert

Added macvlan and ipvlan drivers

GordonTheTurtle added the status/0-triage label Feb 22, 2016

aboch reviewed Feb 23, 2016
View reviewed changes

Added macvlan and ipvlan drivers

1d6f2c5

- Notes at: https://gist.github.com/nerdalert/c0363c15d20986633fda Signed-off-by: Brent Salisbury <brent@docker.com>

mavenugo reviewed Mar 2, 2016
View reviewed changes

nerdalert mentioned this pull request Mar 4, 2016

Address already in use for IPv6 Local-Link #988

Closed

mavenugo mentioned this pull request Mar 6, 2016

Allow networks to overlap subnet (using option) moby/moby#20986

Closed

mavenugo reviewed Mar 7, 2016
View reviewed changes

Make macvlan and ipvlan drivers as experimental

382aeea

Signed-off-by: Madhu Venugopal <madhu@docker.com>

aboch added a commit that referenced this pull request Mar 10, 2016

Merge pull request #964 from nerdalert/ipvlan_macvlan

53a6d24

Added macvlan and ipvlan drivers

aboch merged commit 53a6d24 into moby:master Mar 10, 2016

mavenugo mentioned this pull request Mar 11, 2016

Vendor libnetwork v0.7.0-dev.7 : Experimental MacVlan and IPVlan network drivers moby/moby#21122

Merged

pierrecdn mentioned this pull request Apr 25, 2016

Routing, IPv6, secondary IFs, trafic control, tunelling trial... jpetazzo/pipework#122

Open

arkodg mentioned this pull request Jan 6, 2020

Internal macvlan network doesn't work in swarm #2418

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added macvlan and ipvlan drivers #964

Added macvlan and ipvlan drivers #964

nerdalert commented Feb 22, 2016

aboch Feb 23, 2016

nerdalert Feb 23, 2016

mavenugo commented Feb 23, 2016

aboch Feb 23, 2016

aboch Feb 23, 2016

mavenugo Mar 2, 2016

mavenugo commented Mar 2, 2016

aboch commented Mar 2, 2016

mavenugo commented Mar 3, 2016

mavenugo commented Mar 3, 2016

nerdalert commented Mar 7, 2016

mavenugo Mar 7, 2016

nerdalert Mar 7, 2016

mavenugo commented Mar 7, 2016

mavenugo commented Mar 7, 2016

fredhsu commented Mar 7, 2016

nerdalert commented Mar 8, 2016

nerdalert commented Mar 8, 2016

mavenugo commented Mar 8, 2016

aboch commented Mar 9, 2016

chenchun commented Mar 10, 2016

Added macvlan and ipvlan drivers #964

Added macvlan and ipvlan drivers #964

Conversation

nerdalert commented Feb 22, 2016

aboch Feb 23, 2016

Choose a reason for hiding this comment

nerdalert Feb 23, 2016

Choose a reason for hiding this comment

mavenugo commented Feb 23, 2016

aboch Feb 23, 2016

Choose a reason for hiding this comment

aboch Feb 23, 2016

Choose a reason for hiding this comment

mavenugo Mar 2, 2016

Choose a reason for hiding this comment

mavenugo commented Mar 2, 2016

aboch commented Mar 2, 2016

mavenugo commented Mar 3, 2016

mavenugo commented Mar 3, 2016

nerdalert commented Mar 7, 2016

mavenugo Mar 7, 2016

Choose a reason for hiding this comment

nerdalert Mar 7, 2016

Choose a reason for hiding this comment

mavenugo commented Mar 7, 2016

mavenugo commented Mar 7, 2016

fredhsu commented Mar 7, 2016

nerdalert commented Mar 8, 2016

nerdalert commented Mar 8, 2016

mavenugo commented Mar 8, 2016

aboch commented Mar 9, 2016

chenchun commented Mar 10, 2016