macvlan subinterface documentation does not work on Ubuntu 16.04

[BlinkyStitt]

File: engine/userguide/networking/get-started-macvlan.md, CC @mistyhacks

I have been using the macvlan adapter with great success for awhile now on Ubuntu 16.04 with Kernel 4.4.0. Today however, I needed my host and my container able to communicate.

Underneath "Communication with the Docker host over macvlan", it says:

A macvlan subinterface can be added to the Docker host, to allow traffic between the Docker host and containers. The IP address needs to be set on this subinterface and removed from the parent address.

This sounds like exactly what I need, but the command ip link add mac0 link $PARENTDEV type macvlan mode bridge is not enough for my system (Ubuntu 16.04). I still need commands for moving the IP address to the subinterface.

This blogpost and moby/libnetwork#2008 and this question had more commands than the official docs, but I was still unable to ping after running them:

sudo ip link add mac0 link $PARENTDEV type macvlan mode bridge
sudo ip addr add $PARENTIP/24 dev mac0
sudo ifconfig mac0 up

This blogpost looks the most complete with some route flushes, but it also failed to work for me.

What am I missing? ip route flush and ip route add maybe?

For now I am just going to edit /etc/network/interfaces like the docs suggest and see how that goes.

Also, there is a minor typo at the end of that section. "For more on Docker networking commands, see Working with Docker network commands](/engine/userguide/networking/work-with-networks/)." -> "For more on Docker networking commands, see Working with Docker network commands."

[kimi1987]

Hi, can you share your host routing table.

[BlinkyStitt]

This is my host's default routes:

# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.11.12.1      0.0.0.0         UG    0      0        0 eno2
10.11.12.0      0.0.0.0         255.255.255.0   U     0      0        0 eno2
10.147.17.0     0.0.0.0         255.255.255.0   U     0      0        0 zt1
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
172.18.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-e092d5b55888
172.19.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-afa63818ce28
172.20.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-e80011374fbe
172.21.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-1642511dee36
172.22.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-a9dcd6de1dac
172.23.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-c2979916b1dc
172.24.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-f5334fdff459
172.25.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-16a7fb076837
172.26.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-1679881c3cd0
172.27.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-1e2e0c1ceffc
172.28.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-5a65601bfd95
192.168.0.0     0.0.0.0         255.255.0.0     U     0      0        0 br-e004298fb939

The container I want the host to be able to talk to is at 10.11.12.144.

[BlinkyStitt]

The documented /etc/network/interfaces works for me, but since the mac0 network has a unique MAC each boot, it gets a new DHCP address each boot. I wonder if we should delete the current ip link add mac0 ... command or add to it.

Working routes:

# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         10.11.12.1      0.0.0.0         UG    0      0        0 mac0
10.11.12.0      0.0.0.0         255.255.255.0   U     0      0        0 mac0
10.147.17.0     0.0.0.0         255.255.255.0   U     0      0        0 zt1
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 docker0
172.18.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-e092d5b55888
172.19.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-afa63818ce28
172.20.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-e80011374fbe
172.21.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-1642511dee36
172.22.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-a9dcd6de1dac
172.23.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-c2979916b1dc
172.24.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-f5334fdff459
172.25.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-16a7fb076837
172.26.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-1679881c3cd0
172.27.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-1e2e0c1ceffc
172.28.0.0      0.0.0.0         255.255.0.0     U     0      0        0 br-5a65601bfd95
192.168.0.0     0.0.0.0         255.255.0.0     U     0      0        0 br-e004298fb939

I have changed my /etc/network/interfaces to:

auto eno2
iface eno2 inet manual

auto mac0
iface mac0 inet dhcp
  pre-up   ip link add mac0 link eno2 address 42:bd:cc:b3:34:f5 type macvlan mode bridge
  pre-down ip link del mac0 link eno2 type macvlan mode bridge

Note the addition of the mac address.

[mdlinville]

@vidurous said he might be able to help update the docs with a more accurate example.

[BlinkyStitt]

Awesome thanks. I've got my system working after that slight modification to /etc/network/interfaces. Using pre-up/pre-down is a much better pattern then the other blogposts were using.

[BlinkyStitt]

We should also add a note about IPv6. I know needs to be disabled on the parent interface or else it will grab an address, but I think I'm missing routes still.

I added this to /etc/sysctl.d/99-local.conf, but it isn't enough:

net.ipv6.conf.eno2.disable_ipv6=1
net.ipv6.conf.mac0.disable_ipv6=0
net.ipv6.conf.mac0.accept_ra = 2

Before I had IPv6 working with:

net.ipv6.conf.eno2.accept_ra = 2

I'm just happy that IPv4 host <-> container is working for my purposes so for now I'm disabling IPv6.

[vidurous]

The following ifcfg network interface adapter configurations have worked well concerning macvlans on RHEL variants and should work on Ubuntu as well.

Macvlan 802.1q Trunk Bridge Mode example usage
https://docs.docker.com/engine/userguide/networking/get-started-macvlan/#macvlan-8021q-trunk-bridge-mode-example-usage

You'll need to create parent/slave interfaces specific to each vlan ontop of eth0 or whatever your network adapter is named. Something to the effect of eth0.1, eth0.2 where eth0.1 is vlan 1 and eth0.2 is vlan 2. You would also need to ensure the gateway lives on both vlan networks as well. Ensure to replace XXX.XXX.XXX.XXX fields with your specific values.

Example configuration:

# cat ifcfg-eth0

BOOTPROTO=none
DEFROUTE=no
PEERDNS=no
PEERROUTES=no
NAME=eth0
ONBOOT=yes
DNS1=<DNS1>
DNS2=<DNS2>
USERCTL=no
NM_CONTROLLED=no


# cat ifcfg-eth0.1

TYPE=Ethernet
BOOTPROTO=none
DEFROUTE=no
PEERDNS=no
NAME=eth0.1
DEVICE=eth0.1
ONBOOT=yes
IPADDR=XXX.XXX.XXX.XXX
NETWORK=XXX.XXX.XXX.XXX
NETMASK=XXX.XXX.XXX.XXX
GATEWAY=XXX.XXX.XXX.XXX
USERCTL=no
NM_CONTROLLED=no
VLAN=yes


# cat ifcfg-eth0.2

TYPE=Ethernet
BOOTPROTO=none
DEFROUTE=no
PEERDNS=no
NAME=eth0.2
DEVICE=eth0.2
ONBOOT=yes
IPADDR=XXX.XXX.XXX.XXX
NETWORK=XXX.XXX.XXX.XXX
NETMASK=XXX.XXX.XXX.XXX
GATEWAY=XXX.XXX.XXX.XXX
USERCTL=no
NM_CONTROLLED=no
VLAN=yes

[praparn]

Hmm. Not surly about environment that you facing problem. But normally if you running macvlan feature on public cloud provider that will denied this communicate as baseline. And also if you running on your on-premise farm (Such as ESXi, OpenStack etc). You also need to set "promiscuous mode" before you operate on Host (Docker Host) inside

[mdlinville]

Macvlan docs were written in #5936 so take a look. Also, yes, if you are on a cloud provider you probably can't use Macvlan. If you are still running into trouble, consider asking on https://forums.docker.com/. Thanks!

[BlinkyStitt]

Thank you!

…

On Feb 7, 2018, at 11:36 AM, Misty Stanley-Jones ***@***.***> wrote: Macvlan docs were written in #5936 so take a look. Also, yes, if you are on a cloud provider you probably can't use Macvlan. If you are still running into trouble, consider asking on https://forums.docker.com/. Thanks! — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[docker-robott]

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

/lifecycle locked