The connections attribute of devices

[MrEbbinghaus]

Background

There are four pre-defined connection types in Home Assistant: mac, zigbee, bluetooth and upnp (see device_registry.py)

These connections are used to identify and merge devices across integrations, which is an important usability feature.

Issue 1 - Naming

The naming of the types is misleading.

Connections of type mac and bluetooth expect 48bit mac addresses.
zigbee expects a 64bit address.

bluetooth and mac are both EUI-48 addresses, they share the same "number space", so there shouldn't be a differentiation between the two. Two integrations that report ("mac", "aa:bb:cc:dd:ee:ff") and ("bluetooth", "aa:bb:cc:dd:ee:ff") therefore report the same device, the type of interface is irrelevant.

zigbee is an EUI-64 address and is recommended by the IEEE to be used by "new" applications. Most notably, this includes Zigbee and Thread.

From Use of EUI-64 for New Designs:

Because of the high potential use rate, the RAC believes that all new designs which use a specific world-wide unique address SHALL use the 64-bit address space (EUI-64) and those applications which are currently on a path to use up all the address space currently using the 48-bit address SHALL migrate to the 64-bit space.

Issue 2 - Normalization & Validation

Of the connection types, only mac has any form of normalization. It turns the formats: xx-xx-xx-xx-xx-xx, xxxx.xxxx.xxxx and xxxxxxxxxxxx) into the form: xx:xx:xx:xx:xx:xx.

If the provided address doesn't match any of the format, it gets passed through. The types bluetooth and zigbee don't have any normalization. This isn't an issue from an UX standpoint, as it can be expected that a malformed mac address simply won't match. In the worst case, an integration reports a malformed, possibly huge, mac address that bloats the registry and breaks the UI.

Other than that there is a slightly more practical issue: Not every address is suitable as an identifier.

Addresses beginning with 00:00:00:xx:xx:xx(:xx:xx) and group addresses (the least significant bit of the first octet = 1) can't be used to identify individual addresses: See
IEEE Guidelines for Use of Extended Unique Identifier (EUI), Organizationally Unique Identifier OUI), and Company ID
(CID)

'Values based on a zero-valued OUI [...] shall not be used as identifiers.'

This became an issue recently, where some Matter devices reported 00:00:00:00:00:00 as their mac address (see home-assistant/core#121257 (comment)).

Proposition

I propose that the addresses should at least be validated to be suitable as identifiers. (see this code review home-assistant/core#121257 (comment)) Especially, NULL and group addresses have the potential to wrongfully merge a lot of devices into one, leading to a horrendous experience.

---

I haven't opened an issue (or PR) for this, because the merging of devices and inclusion of entities is a large ongoing UX issue. Blog entry for 2024.7 and this related forum post: Why can’t we add template entities to existing devices?.
I wanted to hear more opinions on how this should be handled going forwards.

[agners]

I don't have a very strong opinion/good idea, but here are some thoughts from my side:

There are four pre-defined connection types in Home Assistant: mac, zigbee, bluetooth and upnp (see device_registry.py)

The connection types are largely physical addresses of network interfaces (cable or radio). From how I understand the connection property is meant to reflect physical connectivity between devices.

For hard-wired network this works obviously quite well, as these connections are rather static.

For Zigbee it's a bit more tricky. Technically, the connection could really represent the mesh. But this likely changes constantly. I guess this is more logically used: All Zigbee devices are part of a network connected through a single controller. So this can be seen as the logical "next hop" in the connection.

For wireless devices it gets a bit more tricky, especially if there are multiple WiFi access points/Thread border router: Which WiFi AP/Thread border router is a wireless end device associated with? Furthermore with Thread there is also the discussion above, should the mesh be represented in the connection?

Anyhow, I probably overthink this. The question really becomes what the goal of tracking this connections is exactly.

These connections are used to identify and merge devices across integrations, which is an important usability feature.

So this is kinda an interesting observation: If it is mainly about identification of a device, why do we even have connection tuples? We could just identify devices based on MAC. A device an also have multiple MACs (e.g. if it has wireless and wired connectivity).

I propose that the addresses should at least be validated to be suitable as identifiers. (see this code review home-assistant/core#121257 (comment)) Especially, NULL and group addresses have the potential to wrongfully merge a lot of devices into one, leading to a horrendous experience.

In a way, this is an unrelated issue. But yes, I agree, it makes sense to validate these MACs and only accept valid ones 👍 . Furthermore, integrations should also be careful that what they provide are indeed per-device unique MAC addresses.

[MrEbbinghaus]

For wireless devices it gets a bit more tricky, especially if there are multiple WiFi access points/Thread border router: Which WiFi AP/Thread border router is a wireless end device associated with? Furthermore with Thread there is also the discussion above, should the mesh be represented in the connection?

and

If it is mainly about identification of a device, why do we even have connection tuples? We could just identify devices based on MAC. A device an also have multiple MACs (e.g. if it has wireless and wired connectivity).

Are you perhaps mixing this problem with the via_device problem we talked about earlier?

The connections attribute is about the connection between an entity and a device in the registry, not physical connections between physical devices.
(identifier is used to group inside a single integration)

---

Although I would say the docs are unclear about this.

identifiers:
Set of (DOMAIN, identifier) tuples. Identifiers identify the device in the outside world. An example is a serial number. Each item in the set uniquely defines a device entry, meaning another device can't have the same identifier.

An example is a serial number.

What's the DOMAIN for a serial number? It's not exclusive to an integration.

Also, the wording for identifiers and connections:

Each item in the set uniquely defines a device entry

is wrong, or at least ambiguous.
Each item in the set refers to the same unique device, but it could be understood that each item defines a unique device entry.

[agners]

Are you perhaps mixing this problem with the via_device problem we talked about earlier?

The connections attribute is about the connection between an entity and a device in the registry, not physical connections between physical devices. (identifier is used to group inside a single integration)

Right, seems I mixed that up. I am not that familiar with the Core I must say 🙈 In a way, it sounds to me that connections is then a bit misleading no? I mean an entity and a device are not connected per se, they are the same device. I guess the naming stems from the fact that a connection interface is used to identify. Maybe interfaces or even interface_identifiers would be less confusing.

Although I would say the docs are unclear about this.

identifiers:
Set of (DOMAIN, identifier) tuples. Identifiers identify the device in the outside world. An example is a serial number. Each item in the set uniquely defines a device entry, meaning another device can't have the same identifier.

An example is a serial number.

What's the DOMAIN for a serial number? It's not exclusive to an integration.

Yeah domains are not exclusive per integration. Not sure what the implications of this are, would two entries with the same identifier get merged? 🤔

Each item in the set uniquely defines a device entry

is wrong, or at least ambiguous. Each item in the set refers to the same unique device, but it could be understood that each item defines a unique device entry.

Yeah, if it works that way, then agreed, this can be misunderstood.

[agners]

zigbee is an EUI-64 address and is recommended by the IEEE to be used by "new" applications. Most notably, this includes Zigbee and Thread.

From Use of EUI-64 for New Designs:

Because of the high potential use rate, the RAC believes that all new designs which use a specific world-wide unique address SHALL use the 64-bit address space (EUI-64) and those applications which are currently on a path to use up all the address space currently using the 48-bit address SHALL migrate to the 64-bit space.

Btw, in Thread the EUI-64 is also named extended address (or ExtAddress). However, for privacy reason the physical EUI-64 addresses are typically not used for communication, but just random ones are generated whenever the Thread network is setup/coonfigured.

Per Thread Specification, the IEEE 802.15.4 Extended Address is randomly generated when the Thread device is commissioned into a Thread network.

From this discussion: https://github.com/orgs/openthread/discussions/9979

In general, with Matter/Thread we see a strong push trying to prevent tracking devices across device resets/commissioning instances. This is probably mostly driven by Google/Apple's push to protect users privacy lately. I think it is a good thing, it is just something we need to keep in mind.

[MrEbbinghaus]

That doesn't mean Matter is reporting such a privacy preserving address.

Thread may use random mac addresses, since there is a concern that someone is sniffing on the medium. (aka listening too signals in the air) (Your neighbour, the suspicious van in front of your house, a Google Street View car...)

Matter isn't concerned about blasting this information over the air and only gives out this information inside a network, where there is little to no need to hide the original address. Especially when most Matter devices aren't moving from untrustworthy network to network, like a phone would on public hotspots.

(Disclaimer: I'm nowhere near an expert regarding Thread or Matter, so I've no idea, what they might or might not be doing.)

Also, you should be able to check if the EUI might be the "real" one, based on the universal/local bit. The second to last bit in the first octet is set if it is locally administered (see Extended Local Identifier (ELI) in the Guidelines).
So every address that begins with X2:XX:.., X6:, XA: or XE: is a locally administered unicast address, while X0:, X4:, X8:, XC: are universally administered.

[agners]

That doesn't mean Matter is reporting such a privacy preserving address.

True, but I would guess that typical platform implementation just read out the current EUI-64/extended address and pass it to the Matter stack. 🤷‍♂️

Thread may use random mac addresses, since there is a concern that someone is sniffing on the medium. (aka listening too signals in the air) (Your neighbour, the suspicious van in front of your house, a Google Street View car...)

Matter isn't concerned about blasting this information over the air [...]

Yeah, but only encrypted, of course. Otherwise what would be the point if the lower layer randomizes it 😅

[...] and only gives out this information inside a network, where there is little to no need to hide the original address. Especially when most Matter devices aren't moving from untrustworthy network to network, like a phone would on public hotspots.

[agners]

I mean, there are devices, so we can just check 😄 So I went ahead and checked a couple of devices:

• Expose universal administered EUI-64 via Matter: Aqara
• Expose locally administered EUI-64 via Matter: Eve, Nanoleaf

And I also re-commissioned an Aqara and an Eve device. Aqara kept showing the same universal administered EUI-64 address, whereas Eve showed a new EUI-64 after commissioning.

Finally, I checked what these devices are using to communicate with. For that I made sure the devices are on my OTBR, and checked their extended address using ot-ctl child list, ot-ctl child <number>. It turns out, that Eve and Nanoleaf do communicate with the locally administered EUI-64 as shown through Matter (expected), and Aqara uses a locally administered EUI-64 which I haven't seen before.

So, to conclude, Aqara behaves how you would expect it. Nanoleaf/Eve is what I expected. So we have both situations in the wild 🙈

[MrEbbinghaus]

Which isn't an issue, as long as there are no collisions, which is highly unlikely. 🙂