Moving SD card between RaspberryPi also takes Mac address?

[charlietomo]

Long story short, I have a production raspberry pi that got a corrupt SD card. It was running Hassio (resin version) and I saw the advice was to move to the new HassOS. On a spare raspberry pi and a brand new SD card I install HassOS, setup a few things (e.g. downloaded required addons).

My thinking was that I would then swop the SD card into the production machine, it would come back online and I could complete the config.

I set static DHCP via DHCP reservations on my router. My prod box is set to .15 and my test box to .20

My issue is that after swopping the sd card from test to prod, it is now coming up on .15 IP address - i.e. it seems to have taken the mac address of the test machine!?

I might be going crazy, but is this intended behaviour - does some of the docker/resin/buildroot magic take the IP from the initial device and spoof that on any future devices? I understood mac address was tied to a physical device?

At the moment I'm a bit stuck because many things are set to that prod IP, so I don't want to change it, and I still want to use the test pi with the same MAC address!

Edit: Not sure if it is related to release 1.3 which mentions:

improvment: Change dhcp client for store static lease / Remember IP on DHCP

[charlietomo]

By way of update. My hypothesis was correct; somehow the os/sd card stores the Mac address it was installed on (!).

I removed the SD card, etched the image again, put it in the prod machine and it setup a new hassio and was visible on the correct IP; i.e. the install process worked out the Mac address of the hardware.

Would love to know where the Mac address is stored and where I could have changed it rather than starting again!

[pvizeli]

That is managed by NetworkManager

[charlietomo]

Understand you have closed this, but regardless of a feature being managed by another software package, if said feature is different from both previous hassio installations AND from conventional IT/networking conventions it strikes me it would be worth pointing out. That could be to external documentation, but so far I haven't found anything that, in simple terms, explains the mac address cloning.

[Curtis-Fletcher]

I just had two devices with identical MACs because of Hassos MAC persistence, took quite a while to faultfind. I'd suggest that if NetworkManager is persisting MAC addresses then it's configured incorrectly in the Hassos image.

If you're building a Network manager profile then saving it perhaps using this:

nmcli connection modify id XXXX 802-3-ethernet.mac-address ''

To remove the MAC from the profile. or perhaps:

802-3-ethernet.cloned-mac-address

Is getting set somewhere in your creation scripts.

[jvanderneutstulen]

Neither Hassos nor NetworkManager will change/clone/store any mac address. (tested with ova build 2.5)
By default the cloned-mac-address is not specified (see default profile in https://github.com/home-assistant/hassos/blob/dev/Documentation/network.md, so the mac will be untouched.

NetworkManager does persist its dhcp lease, so when Hassos boots it will request the same ip address it had before. It depends on the dhcp-server if it actually will get the requested address.

In my case with a Mikrotik router, the dhcp server will send a NAK when the original lease is stlll valid, after which NetworkManager will perform a dhcp discover and gets a different address. When the original lease has expired it will assign the requested ip. When original lease was a static/reserved lease, it will assign a different ip address.

If you want to clear the old lease info, you remove the lease info via a debug login from /var/lib/NetworkManager.

[Curtis-Fletcher]

Two Raspberry Pi's, lets call them RPIHASSOS and RPITEST

DHCP Static assignments at DHCP server:

B8:27:EB:DF:64:5D		192.168.1.198		hassio

Boot RPIHASSOS that Hassos initially booted from, running Hassos
login on console as root (after modifying squash FS to gain root access to investigate)

# cat /sys/class/net/eth0/address
b8:27:eb:df:64:5d

Shutdown, Swap SD card, reboot
Boot RPIHASSOS, running Rasberian

pi@raspberrypi:~ $ cat /sys/class/net/eth0/address
b8:27:eb:df:64:5d

pi@raspberrypi:~ $ ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST     mtu 1500
        inet 192.168.1.198  netmask 255.255.255.0  broadcast 192.168.1.255
        inet6 fe80::5cc7:a135:4939:e67d  prefixlen 64  scopeid 0x20<link>
        ether b8:27:eb:df:64:5d  txqueuelen 1000  (Ethernet)
        RX packets 108  bytes 14201 (13.8 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 105  bytes 18056 (17.6 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

etc, Shutdown
Boot RPITEST with same Rasberian SD card

pi@raspberrypi:~ $ cat /sys/class/net/eth0/address
b8:27:eb:52:68:95

pi@raspberrypi:~ $ ifconfig
eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST     mtu 1500
        inet 192.168.1.51  netmask 255.255.255.0  broadcast 192.168.1.255
        inet6 fe80::2e39:5bb5:170f:2664  prefixlen 64  scopeid 0x20<link>
        ether b8:27:eb:52:68:95  txqueuelen 1000  (Ethernet)
        RX packets 102  bytes 13682 (13.3 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 88  bytes 12859 (12.5 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Shutdown, Swap SD card, reboot
Boot RPITEST running Hassos

# cat /sys/class/net/eth0/address
b8:27:eb:df:64:5d

Reading more about NetworkManager I see that the Hassos file:

/etc/NetworkManager/system-connections/default

contains the line:

addr-gen-mode=stable-privacy

But that is IPv6, so it shouldn't affect IPv4

However I see interesting information in the Networkmanager settings under "stable-id"

stable-id

string

This represents the identity of the connection used for various purposes. It allows to configure multiple profiles to share the identity. Also, the stable-id can contain placeholders that are substituted dynamically and deterministically depending on the context. The stable-id is used for generating IPv6 stable private addresses with ipv6.addr-gen-mode=stable-privacy. It is also used to seed the generated cloned MAC address for ethernet.cloned-mac-address=stable and wifi.cloned-mac-address=stable. It is also used as DHCP client identifier with ipv4.dhcp-client-id=stable and to derive the DHCP DUID with ipv6.dhcp-duid=stable-[llt,ll,uuid]

Which suggests there may be cross-polination between that IPv6 option and the persisting IPV4 MAC address that I am seeing.
It's a shame ip/ifconfig aren't present in the image, there might be something interesting there.

I'm new to NetworkManager (I have history with BSD and Linux but I've never used/needed it) But as you can see it is happening. I guess my next port of call it to get a tcpdump of the DHCP exchange, see what NetworkManager is instructing the DHCP Client to do.

[Curtis-Fletcher]

I also see:

connection.stable-id, ethernet.cloned-mac-address

If left unspecified, it defaults to "preserve".

Then elsewhere:

permanent - use the native built in MAC address of the device.

preserve - use any MAC address already assigned to the device. If this is not already changed via macchanger, udev or networkd this will be the normal MAC of the device.

[jvanderneutstulen]

@Curtis-Fletcher Thanks for your detailed explanation. However you seem to be running Hass.io on Raspbian, not on Hassos, so it seems to be a Raspbian issue.

That said, the MAC is probably stored in ether /etc/NetworkManager/ or /var/lib/NetworkManager/. Or maybe Raspbian is applying some udev rules.

[Curtis-Fletcher]

However you seem to be running Hass.io on Raspbian, not on Hassos, so it seems to be a Raspbian issue.

I'm afraid you are mistaken. There are two Pi's and two OS installs involved in this testing.

• One is a 4-ish day old release install of hassos (That is persisting the MAC of the Pi it was initially installed on, which is the reason I'm posting here) installed from the "hassos_rpi3-1.13.img" image
• The other is my baseline of a fresh release of Raspbian (I'm using this to verify the actual hardware MAC's of the PI's) installed from the "2018-11-13-raspbian-stretch-lite.img" image

(If you have a look at my log above you should see me noting "shutdown, swap sd"-like actions, this is when I change OS for verification of the actual MAC)

The only change made to the Hassos image is that I extracted the squashfs image for the root FS, altered the root user and re-compressed/wrote-back in order to be able to log into the console. Which was only done after this problem manifested in order to fault-find.

Also I've looked in the mounted /var/lib/NetworkManager/ location and /etc/NetworkManager/ in running Hassos install and currently don't see it. and yet it is happening. I also wonder if the MAC is being sent to NetworkManager via udev

Just for clarity I'll summarize the events listed in my log above

• Boot RPIHASSOS hardware that Hassos image initially booted from, running Hassos SD Card

  • MAC is b8:27:eb:df:64:5d

• Boot RPIHASSOS hardware with baseline Raspbian SD card

  • MAC is b8:27:eb:df:64:5d

• Boot RPITEST hardware, running Hassos SD Card

  • MAC is b8:27:eb:df:64:5d

• Boot RPITEST hardware with baseline Raspbian SD card

  • MAC is b8:27:eb:52:68:95

[charlietomo]

@Curtis-Fletcher thanks for your persistence with this. It's good to know I'm not the only one with this issue. You are delving much deeper than I did - but seems we both wasted a bunch of time identifying the issue. Watching this thread with interest.

[jvanderneutstulen]

@Curtis-Fletcher Can you try with release 2.5 ?
And what model raspi do you have and can you post the output of cat /proc/cpuinfo?

[pvizeli]

Maybe it's a bug inside NetworkManager in that version they we use? I agree we don't set the flag for persistent mac.

Anyway, the root OS is read-only. There is a partition with persistent config files. You can clean up the /mnt/state/var/lib/NetworkManager and they should be reset.

[Curtis-Fletcher]

And what model raspi do you have and can you post the output of cat /proc/cpuinfo?

I don't have them booted right now (because of the MAC issue) but RPIHASSOS was a 3B and RPITEST was a 3B initially then a 3B+ on the second test documented above. Both tests on the 3B and the 3B+ were identical as documented above.

At no point was I using wifi.

[jvanderneutstulen]

Ok, I'm unable to reproduce your issue with some 3B+ devices using a 2.5 image:

First device:

# cat /proc/cpuinfo
..
Hardware	: BCM2835
Revision	: a020d3
Serial		: 00000000e3e59f1b

# cat /sys/class/net/eth0/address 
b8:27:eb:e5:9f:1b

some output from journalctl
Jan 10 20:41:08 hassio dhclient[282]: DHCPREQUEST on eth0 to 255.255.255.255 port 67
Jan 10 20:41:14 hassio dhclient[282]: DHCPREQUEST on eth0 to 255.255.255.255 port 67
Jan 10 20:41:14 hassio dhclient[282]: DHCPACK from 192.168.1.1
Jan 10 20:41:14 hassio dhclient[282]: bound to 192.168.1.183 -- renewal in 1725 seconds.
Jan 10 20:41:15 hassio avahi-daemon[300]: Registering new address record for 2001:x:y:z:25c8:b543:5eb1:7609 on eth0.*.
Jan 10 20:41:15 hassio avahi-daemon[300]: Registering new address record for 192.168.1.183 on eth0.IPv4.
Jan 10 22:22:33 hassio dhclient[282]: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 8
Jan 10 22:22:33 hassio dhclient[282]: DHCPREQUEST on eth0 to 255.255.255.255 port 67
Jan 10 22:22:33 hassio dhclient[282]: DHCPOFFER from 192.168.1.1
Jan 10 22:22:33 hassio dhclient[282]: DHCPACK from 192.168.1.1
Jan 10 22:22:33 hassio dhclient[282]: bound to 192.168.1.183 -- renewal in 1605 seconds.

(time jump because of ntp time change)

After a while shutdown and boot another 3B+ with the same sdcard:

# cat /proc/cpuinfo
..
Hardware	: BCM2835
Revision	: a020d3
Serial		: 000000009979c261

#  cat /sys/class/net/eth0/address 
b8:27:eb:79:c2:61

Again some lines from journalctl:
Jan 10 22:36:02 hassio dhclient[274]: DHCPREQUEST on eth0 to 255.255.255.255 port 67
Jan 10 22:36:07 hassio dhclient[274]: DHCPREQUEST on eth0 to 255.255.255.255 port 67
Jan 10 22:36:07 hassio dhclient[274]: DHCPNAK from 192.168.1.1
Jan 10 22:36:08 hassio dhclient[274]: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 6
Jan 10 22:36:08 hassio dhclient[274]: DHCPREQUEST on eth0 to 255.255.255.255 port 67
Jan 10 22:36:08 hassio dhclient[274]: DHCPOFFER from 192.168.1.1
Jan 10 22:36:08 hassio dhclient[274]: DHCPACK from 192.168.1.1
Jan 10 22:36:08 hassio dhclient[274]: bound to 192.168.1.167 -- renewal in 1505 seconds.
Jan 10 22:36:09 hassio avahi-daemon[293]: Registering new address record for 2001:x:y:z:25c8:b543:5eb1:7609 on eth0.*.
Jan 10 22:36:09 hassio avahi-daemon[293]: Registering new address record for 192.168.1.167 on eth0.IPv4.

Observations: mac address differs, dhcp reject request for same ip on second boot. ipv6 address is stable and remains the same. All as expected.

So, something is different. Maybe the raspi hardware revision matter, or maybe something triggers NetworkManager to change the mac. If you stop NetworkManager, does it change the mac back?

[a-r-j-a-n]

I transfered my sd from a pi3b to a pi3b+ and also have the same issue with the mac address on hassos.

[charlietomo]

Your quickest way to fix might be to backup (in hassos), reinstall hassos from scratch on the new pi, which will use the new pi mac address, and then restore the hassos backup.

[a-r-j-a-n]

Currently doing that now :) 👍 thanks to the great backup/restore functionality, this is the quickest way.