Random MAC ADDRESS on MACVLAN adapter

[Thormir84]

Description

After installing the latest update (5.28.0.0-1 - debian.12), every time I restart or update a container with a MACVLAN adapter associated with it, its MAC ADDRESS changes randomly; the "problem" is that I have LAN monitoring utilities that, every time, notify me of the access of a new device.

Reproduce

Install the latest version of docker.
Reboot or upgrade an existing container, which is associated with a MACVLAN adapter.

Expected behavior

The MAC ADDRESS of the container must always remain the same.

docker version

Client: Docker Engine - Community
 Version:           28.0.0
 API version:       1.48
 Go version:        go1.23.6
 Git commit:        f9ced58
 Built:             Wed Feb 19 22:10:43 2025
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          28.0.0
  API version:      1.48 (minimum version 1.24)
  Go version:       go1.23.6
  Git commit:       af898ab
  Built:            Wed Feb 19 22:10:43 2025
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.7.25
  GitCommit:        bcc810d6b9066471b0b6fa75f557a15a1cbf31bb
 runc:
  Version:          1.2.4
  GitCommit:        v1.2.4-0-g6c52b3f
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

docker info

Client: Docker Engine - Community
 Version:    28.0.0
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.21.0
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.33.0
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 44
  Running: 44
  Paused: 0
  Stopped: 0
 Images: 43
 Server Version: 28.0.0
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: bcc810d6b9066471b0b6fa75f557a15a1cbf31bb
 runc version: v1.2.4-0-g6c52b3f
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.11.11-1-pve
 Operating System: Debian GNU/Linux 12 (bookworm)
 OSType: linux
 Architecture: x86_64
 CPUs: 12
 Total Memory: 62.48GiB
 Name: nas-casa
 ID: 2TGI:O4UR:2ZN7:NAE3:MHND:AYL7:BZAN:V2EJ:N6PC:N7FS:5BJR:TH6Y
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Username: thormir84
 Experimental: false
 Insecure Registries:
  ::1/128
  127.0.0.0/8
 Live Restore Enabled: false

Additional Info

No response

[robmry]

Hi @Thormir84 - yes, that's expected in 28.0.0 (release notes).

You can configure a MAC addresses when creating a container with the --mac-address option.

[Peppercorn27]

This broke some aspects of home assistant for me. My Shelley devices and emulated hue / alexa pairing stopped working. I assume this has do with a mis-matched mac address after recreating my homeassistant container with docker v28.

I fixed this by restoring a snapshot, noting the mac address of the homeassistant container (docker inspect <homeassistant>), then updating to v28 and adding the following to compose:

service:
  homeassiant:
...
    networks:
      IoT_Network:
        ipv4_address: ${HOMEASSISTANT_IP}
        mac_address: xx:xx:xx:xx:xx:xx # your mac address
...

[gaby]

@robmry A macvlan device to be changing mac-addresses every time the container is updated/restarted is a very destructive change. We have internal health-checks to auto-kick the container when the internet connection is misbehaving with this change the mac-address is now being changed randomly. This behavior in docker has been solid for years, now out of nowhere in v28.x.y and bunch of networking changes were introduced all at once.

The mac-address should only change when the container is created, not during a restart/update.

So far my team has run into 5-6 massive breaking changes in v28.x.y related to networking for things that have been stable for several years with docker/libnetwork.

List of breaking changes so far:

• The mac-addresses are now randomly changing.
• Containers with multiple networks, are now getting their interfaces created in random-order.
• The priority and gw-priority are ignored for ipvlan, macvlan and external networks.
• [Fixed in 28.0.1] Random rules being added to FORWARD chain made us have a 4hr outage.
• [Fixed in 28.0.1] The ip6tables being enforced when ipv6=false is offline.

[robmry]

Hi @gaby - thanks for letting me know.

We try to minimise the impact of any changes but sometimes, in a major release, they are unavoidable.

The mac-addresses are now randomly changing.

In previous releases, a container's MAC address was based on its IPv4 address. From 28.x, a container may no longer have an IPv4 address (or, in the case of macvlan, any IP address at all). And, a MAC address can't be derived from an IPv6 address in the same way because the MAC address doesn't have enough bits. The mapping between addresses was also a blocker for adding IPv6 support to Swarm.

So, we had to break the relationship between IPv4 address and MAC address.

In all releases, when a container is stopped it loses its IP address reservation - another container can be started with the same IP address while it's stopped. If the IPv4 address is explicitly configured, but gets reused while the container is stopped, the container won't start because of the clash. If the IPv4 address is not explicitly configured, the container will start with a new IP address and, in 27.x and earlier, with a new MAC address matching that IP address. (Neither IP or MAC address are reserved while a container is stopped, and both may be automatically assigned new values on a container restart or when a container is re-created, unless they're explicitly configured.)

So, breaking the link between IPv4 and MAC addresses means when a container with no configured MAC address is restarted in 28.x, the address assigned when it restarts is randomly generated - so, it will change, even when the IP address is configured.

It may be possible to preserve a container's generated MAC address while it's stopped, so it doesn't change on a docker restart. But, I don't think that would help much ... for example, compose down/up re-creates containers, they're deleted and added again. Docker Engine doesn't know anything about the relationship between the before-and-after containers. If they're configured to have a specific IP address, they'll get that address again, likewise for a MAC address. Trying to preserve generated MAC addresses in that way would also lead to surprising changes on a docker re-install, re-creating the service would generate new MAC addresses and the old addresses would need to be configured-in at that point.

So, as with IP addresses - if a specific MAC address is significant to other services in the network, the way to make sure it doesn't change is to configure it.

Containers with multiple networks, are now getting their interfaces created in random-order.

I think that was the case in 27.x too. The moby API uses a map of network endpoints (rather than an ordered list). Do you have an example of the changed behaviour?

Is the issue that the container's interface names (eth0 etc) belong to different networks after a restart? In 28.x, there's a new option to explicitly name network interfaces (release notes). Perhaps that will help?

The priority and gw-priority are ignored for ipvlan, macvlan and external networks.

Can you share details or an example? ...

The priority field in compose hasn't changed. Since compose 2.24 (Feb '24), it has not influenced the container's network device names, and it has never influenced which network provides the default gateway.

In 28.x, the new gw-priority can be used to determine which network provides the default gateway. It does not consider which network driver the network is being used. But, a dual-stack (IPv4 and IPv6) network is always preferred over single-stack as the gateway - perhaps that's what you've seen?

[gaby]

@robmry Sorry for the late response.

For the first case I probably didn't run into this because the container has a /29 or /31 network. But your explanation about decoupling makes sense.

I was able to mitigate the network order by using the new opt. For some reason this was not a problem in v27.x but i'm launching the containers with compose so it could be related to that.

The macvlan network ignoring the `gw-priority is also launched via compose. There's a change the bug is there and not in docker itself.

So far with 28.0.1 and 28.0.2 we had no issues.

[robmry]

Thanks @gaby - I'm glad the patch releases are behaving themselves.

If there's still an issue with macvlan and the gateway priority and you'd like us to take a look, please do raise a new issue (in the moby project, https://github.com/moby/moby/issues).

[Thormir84]

I thought the problem was solved using the string "mac_address: xx:xx:xx:xx:xx:xx", but I noticed that, every time a container is automatically updated (using Watchtower, in my case), the problem recurs; When that happens, I have to stop and restart the stack manually, with no changes, and everything goes back to the way it was before (until the next update).

[robmry]

Hi @Thormir84 ... so, if you create or re-create a container with a configured mac address using Compose, the mac address is applied correctly. (I guess you're using a real address, not literally x's?)

But, when you re-create the container using Watchtower, it doesn't configure the mac address?

Compose and Watchtower use the same engine API to create containers - so there must be a difference in how Watchtower is using it, or perhaps in how it reads/interprets the config of the old container.

If you enable debug logging over a failed Watchtower upgrade, and send the logs along with the compose config, I can take a look at the API requests. https://docs.docker.com/engine/daemon/logs/#enable-debugging

[Thormir84]

Technically the upgrade does not fail; it is as if the string relating to the mac address was ignored (obviously I don't use "xx" as a mac address).
I noticed that, from Portainer, if I stop and restart the stack or press the "update the stack" button (without changing anything), the mac address is applied correctly. Otherwise, if I do a simple "restart" (always from Portainer), every time I find a different mac address.
This doesn't happen if I restart the host machine or if I update the docker engine.

[robmry]

I think it's unlikely docker is ignoring a MAC address in a container create request, and I believe a configured MAC address is preserved over a docker restart.

But, if you send the debug logs, I'll take a look.

[robmry]

The macvlan network ignoring the `gw-priority is also launched via compose. There's a change the bug is there and not in docker itself.

I think it was compose ... docker/compose#12574