Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bridge: Fix hwaddr set race between us and udev #2380

Merged
merged 1 commit into from Jan 1, 2020

Commits on May 19, 2019

  1. bridge: Fix hwaddr set race between us and udev

    systemd and udev in their default configuration attempt to set a
    persistent MAC address for network interfaces that don't have one
    already [systemd-def-link]. We set the address only after creating the
    interface, so there is a race between us and udev. There are several
    outcomes (that actually occur, this race is very much not a theoretical
    one):
    
    * We set the address before udev gets to the networking rules, so udev
      sees `/sys/devices/virtual/net/docker0/addr_assign_type = 3`
      (NET_ADDR_SET). This means there's no need to assign a different
      address and everything is fine.
    
    * udev reads `/sys/devices/virtual/net/docker0/addr_assign_type` before
      we set the address, gets `1` (NET_ADDR_RANDOM), and proceeds to
      generate and set a persistent address.
    
      Old versions of udev (pre-v242, i.e. without [udev-patch]) would then
      fail to generate an address, spit out "Could not generate persistent
      MAC address for docker0: No such file or directory" (see [udev-issue],
      and everything would be probably fine as well.
    
      Current version of udev (with [udev-patch]) will generate an address
      just fine and then race us setting it. As udev does more work than we,
      the most probable outcome is that udev will overwrite the address we
      set and possibly cause some trouble later on.
    
    On a clean Debian Buster (from Vagrant) VM with systemd/udev 242 from
    Debian Experimental, `docker network create net1` up to `net7` resulted
    in 3 bridges having a 02:42: address and 4 bridges having a seemingly
    random (actually generated from interface name) address. With systemd
    241, the result would be all bridges having a 02:42:, but some "Could
    not generate persistent MAC address for" messages in the log.
    
    The fix is to revert the MAC address setting fix from 79b3e77,
    as it is no longer necessary with current netlink [netlink-addr-add],
    and set the address atomically when creating the bridge interface, not
    after that.
    
    [systemd-def-link]: https://github.com/systemd/systemd/blob/a166cd3aacdbfd4df196bb4ca9f43cff19cf9fec/network/99-default.link
    [udev-patch]: systemd/systemd@6d36464
    [udev-issue]: systemd/systemd#3374
    [netlink-addr-add]: vishvananda/netlink@7d9b424
    
    ...
    
    Do note that a similar race happens when creating veth devices as well.
    I wasn't able to reproduce getting a wrong (non-02:42:) address,
    possibly because the address is set by docker later, maybe only after
    the interface is moved to another network namespace (but I'm just
    guessing here). Still, different timings result in various error
    messages being logged ("link_config: could not get ethtool features for
    vethd9c938e" and the like) depending on when the interface disappears
    from the primary network namespace. I'm not sure how to fix this and I
    don't intend to dig deeper into this.
    
    Signed-off-by: Tomas Janousek <tomi@nomi.cz>
    liskin committed May 19, 2019
    Copy the full SHA
    8710ffe View commit details
    Browse the repository at this point in the history