Add MQTT Sensor unique_id #13318

OttoWinter · 2018-03-18T16:58:20Z

Description:

Add unique_id capabilities to MQTT Sensors. Some sensors (like the ds18b20) have serial numbers that are burnt into the chip at production time. When coupled with MQTT discovery (and esphomelib 🤓) this could allow those sensors to be added to the entity registry.

Very much an RFC and I have a couple of open questions about this:

Should we limit this to the sensor component? I mean over the last week I've tried to think of any other device that exposes truly unique ids, and all of them were sensors.
We should make very clear that we then expect truly unique ids in the documentation. Up until now, the meaning of unique_id has not really been exposed to users, only to developers. This is the main reason I'm not sure about this.

Pull request in home-assistant.github.io with documentation (if applicable): home-assistant/home-assistant.io#5131

Example entry for `configuration.yaml` (if applicable):

sensor:
  - platform: mqtt
    name: Test 1
    state_topic: "test-topic-1"
    unique_id: "TOTALLY_UNIQUE"

Checklist:

The code change is tested and works locally.
Local tests pass with tox. Your PR cannot be merged unless tests pass

If user exposed functionality or configuration variables are added/changed:

Documentation added/updated in home-assistant.github.io

If the code does not interact with devices:

Tests have been added to verify that the new code works.

tinloaf · 2018-03-18T20:21:56Z

Not sure if I understand this right: All this currently does is expose a unique_id config option, that will make the sensor have that unique ID, right? What is the benefit for us (or the user) over just setting the name of the entity?

OttoWinter · 2018-03-18T20:41:04Z

@tinloaf When specifying the devices manually in configuration.yaml there is indeed little use for the unique_id: option. But when using this with MQTT discovery this feature can get quite useful.

For example, esphomelib has support for ds18b20 sensors that all have an individual 64-bit address. In the library I could then send this known address together with the discovery MQTT payload like this:

{  
  "name": "Boiler Temperatur Oben",
  "platform": "mqtt",
  "state_topic": "heatpump/sensor/boiler_temperatur_oben/state",
  "availability_topic": "heatpump/status",
  "unit_of_measurement": "°C",
  "expire_after": 1500,
  "unique_id": "DallasComponent#0x01031663650aff28"
}

The advantage would then be that I don't have to change the entity_id in all my configuration once I change the friendly name on my external MQTT device (+ all the other great entity registry advantages).

tinloaf · 2018-03-18T20:44:39Z

Sure - if unique IDs can be discovered from the devices, I'm all for it. I'm just not sure if it's a good idea to expose this in the config. It adds additional complexity without a benefit (that I can see?). My proposal would be: Add unique IDs where they can be discovered, but don't ask the user to manually configure unique IDs.

OttoWinter · 2018-03-18T20:51:19Z

Yes that was also my initial concern. I like your idea of only enabling it through MQTT discovery 👍 I will take a look at that.

balloob · 2018-03-21T17:24:47Z

So I think that this is fine. Since MQTT is a transport instead of an actual device, we can leave it up to the user to decide what unique ID should be.

To wrap up this PR:

Add a test
Add a comment to the unique id inside voluptuous schema that MQTT is an exception (in case people are going to copy paste things for other integrations)

balloob · 2018-03-21T17:25:59Z

It adds additional complexity without a benefit (that I can see?).

Guaranteed same entity ID because of reserved entity IDs
Customizable entity IDs
Change name via UI while Home Assistant is running
All the other cool stuff we're going to add to entity registry 👍

OttoWinter · 2018-03-21T18:01:00Z

About the test: With the test that I created previously I discovered #13316 - so it was a failing test; I now changed the test a bit so that it still passes and does at least part of the check it's supposed to do.

Should we do something special with the home-assistant.io docs too? Is something like this good as a description:

An id that uniquely identifies this sensor. Only use this if you know what you're doing!

balloob · 2018-03-21T20:46:52Z

Only use this if you know what you're doing!

That's a horrible thing to put in the docs haha. If 2 sensors have the same unique id, Home Assistant will raise an exception.

thepotoo · 2018-03-31T20:34:08Z

This is amazing, really slick!

With the right sketch on the MQTT device, this will allow adding MQTT devices (like Sonoffs and ESP8266) without the need to edit configuration.yaml at all. ESP.getChipId(); can be used to generate a unique_id on the backend.

Could you please also enable this for binary_sensor and switch types?

OttoWinter · 2018-04-04T12:30:51Z

@thepotoo To make clear: This feature should only be used with sensors that have a laser-engraved serial number and only do one thing. ESP.getChipId() does get you a unique ID for the microcontroller, but not for specific functions. For example, this should definitely not be used with binary switches on GPIO pins, since users can change the pins they're using for the same switch, and therefore those IDs would not be unique.

I'll work on this more tomorrow and I'll limit this PR to sensors for now. If someone finds a valid use-case for binary sensors too, they can of course open a PR for that too.

balloob · 2018-04-06T14:14:33Z

This is a PR to add a unique ID to MQTT sensors. Please do not discuss other topics.

thepotoo · 2018-04-06T14:47:36Z

Oh, sorry! I have made a new PR for this, here: #13719

…

On Fri, Apr 6, 2018 at 10:14 AM, Paulus Schoutsen ***@***.***> wrote: This is a PR to add a unique ID to MQTT sensors. Please do not discuss other topics. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#13318 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AeFn9GCERJioaeDqego_KdIOxjtNUfmcks5tl3hZgaJpZM4SvRzG> .

fabaff · 2018-04-06T17:09:50Z

The discovery part already contains the option to set a device ID with <node_id> as part of the topic. Something like home/sensor/xyz1234/temperature/config would lead to an entity ID of xyz1234_temperature. Looks pretty unique to me.

From my point of view should the node_id, if provided, go into the unique_id. It's just a guess but I think that node_id was not only used to simplify the subscription of topics but also to identify devices.

thepotoo · 2018-04-06T17:19:33Z

Is there ever a time when you would want to have a separate entity id and unique id? I can't think of one, so I would fine with generating a unique_id from the node id. The whole point of doing this is to enable me to 1) use one firmware for all devices and 2) be able to rename objects from within the UI as long as I can do that, I'm happy.

…

On Fri, Apr 6, 2018 at 1:10 PM, Fabian Affolter ***@***.***> wrote: The discovery part already contains the option to set a device ID with <node_id> as part of the topic. Something like home/sensor/xyz1234/ temperature/config would lead to an entity ID of xyz1234_temperature. Looks pretty unique to me. For my point of view should the node_id, if provided, go into the unique_id. It's just a guess but I think that node_id was not only used used to simplify the subscription of topics but also to identify devices. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#13318 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AeFn9KCeO4nDgHH3LiDLj5nN5_m60Gaiks5tl6F0gaJpZM4SvRzG> .

OttoWinter · 2018-04-06T17:23:00Z

@fabaff At least speaking for esphomelib, it's not really unique, because the discovery topic is generated based on the name of the entity. So for example if a light is called "Living Room Lights", the object id will be "living_room_lights" for discovery. The problem with using <node_id>_<object_id> for a unique id I think is that discovery users have to specify an object id. Discovery just needs the object id for topic names to work correctly. Unique IDs are supposed to be unique for eternity and not depend on something like the friendly name or user-defined string.

The nice thing about having it as a configuration option is that it's more opt-in. So only the entities that really do have a unique ID (like dallas sensors), will actually receive a unique id. Enabling the unique_id feature for too many integrations will just lead to lots of user confusion.

@thepotoo Just to explain, the problem with switch wasn't the code for me. It was more that I would first like to see how users are using this option. It's always easier to remove an option again from one integration than many at once.

thepotoo · 2018-04-06T17:45:33Z

OK. I have been using this for a couple weeks now and really love the feature. What can I do to help get it implemented for switches?

…

On Fri, Apr 6, 2018 at 1:23 PM, Otto Winter ***@***.***> wrote: @fabaff <https://github.com/fabaff> At least speaking for esphomelib, it's not really unique, because the discovery topic is generated based on the name of the entity. So for example if a light is called "Living Room Lights", the object id will be "living_room_lights" for discovery. The problem with using <node_id>_<object_id> for a *unique* id I think is that discovery users *have* to specify an object id. Discovery just needs the object id for topic names to work correctly. Unique IDs are supposed to be unique for eternity and not depend on something like the friendly name or user-defined string. The nice thing about having it as a configuration option is that it's more opt-in. So only the entities that really do have a unique ID (like dallas sensors), will actually receive a unique id. Enabling the unique_id feature for too many integrations will just lead to lots of user confusion. @thepotoo <https://github.com/thepotoo> Just to explain, the problem with switch wasn't the code for me. It was more that I would first like to see how users are using this option. It's always easier to remove an option again from one integration than many at once. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#13318 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AeFn9AlOE1K3YM-tT3e2iQbE_cfiMno9ks5tl6SHgaJpZM4SvRzG> .

fabaff · 2018-04-06T22:10:31Z

So for example if a light is called "Living Room Lights", the object id will be "living_room_lights" for discovery.

This is just an implementation detail of esphomelib 😉

Unique IDs are supposed to be unique for eternity and not depend on something like the friendly name or user-defined string.

If you combine the address/ID/what-ever (node_id) with a user provided string (object_id) it will be unique forever as one part is unique.

The nice thing about having it as a configuration option is that it's more opt-in.

node_id is opt-in as well but no really used by the platforms.

At the end of the day putting the unique_id in the payload will probably be simpler for most users. The rest will be able to combine it with node_id on the device side anyways.

OttoWinter · 2018-04-06T22:33:36Z

This is just an implementation detail of esphomelib 😉.

Yes, I'm aware of that. I was just using it to hopefully make the point about forcing object_id for the unique_id not being that great of an idea IMO. If there's another way to make it work, I'm sure I would be able to get it to work in esphomelib too. However, there's really not much any system can base truly unique object_ids on other than with serial numbers engraved in sensors AFAIK.

If you combine the address/ID/what-ever (node_id) with a user provided string (object_id) it will be unique forever as one part is unique.

Yes, that unique ID would be unique, but only in one direction (🎵🤦). You're right that every time an unique_id string is generated, that string would be found once in the entity registry. For example, if I rename the object_id from "living_room_lights" to "kitchen_lights", we would never see the same unique_id twice in the entity registry.

But the unique_id property also has to be unique in the other direction as well AFAIK. What I mean by that is that if you have an entity, let's say a light, it should always point to the same unique ID. At least that's my impression from the PRs I've been reading. Otherwise we could also use IP addresses as unique IDs, as they provide a unique mapping from unique id to entity, but not necessarily the other way around.

I mean I only started around the time unique ids were really enabled by the entity registry, so the latter "property" of unique ids (bi-directional uniqueness) is not required.

balloob · 2018-04-07T01:58:59Z

So first off, MQTT is different for unique ID than other platforms. It's different because MQTT topics don't represent devices, it's up to the user to decide what is what. Does the topic represent the name, the value etc.

Because unique ID has such a big impact, we should not try to be smart and auto detect it, we should only allow it to be set via the config. That's also why this PR has the comment in the voluptuous schema to be cautious.

OttoWinter · 2018-04-07T11:07:35Z

Ok, I've updated the code now and created a docs PR: home-assistant/home-assistant.io#5131. Is there something else that I should do?

(I'd still be interested whether unique_id needs to be "bi-directionally" unique as per previous comment, if someone has time 😺)

balloob · 2018-04-08T02:29:50Z

Yeah, it should be bi-directionally. That's why we try to aim to use serial etc. MQTT is a bit more loose because it's up to the user to decide what is a device.

* Add MQTT Sensor unique_id * Add test * Update comment

homeassistant added platform: sensor.mqtt small-pr PRs with less than 30 lines. cla-signed labels Mar 18, 2018

homeassistant added the cla-signed label Mar 21, 2018

OttoWinter changed the title ~~[RFC] Add MQTT Sensor unique_id~~ Add MQTT Sensor unique_id Mar 21, 2018

This comment has been minimized.

Sign in to view

syssi mentioned this pull request Apr 6, 2018

Add unique_id to MQTT switch #13719

Merged

8 tasks

OttoWinter mentioned this pull request Apr 7, 2018

MQTT Sensor Unique ID feature home-assistant/home-assistant.io#5131

Merged

2 tasks

OttoWinter added 2 commits April 7, 2018 13:03

Add MQTT Sensor unique_id

31af74e

Add test

2b8fc59

OttoWinter force-pushed the mqtt-unique-id branch from 7701a91 to 2b8fc59 Compare April 7, 2018 11:05

homeassistant added the cla-signed label Apr 7, 2018

Update comment

f4e8f11

balloob approved these changes Apr 8, 2018

View reviewed changes

balloob merged commit 81b1d08 into home-assistant:dev Apr 8, 2018

balloob mentioned this pull request Apr 27, 2018

0.68 #14124

Merged

ciotlosm mentioned this pull request May 28, 2018

Allow unique_id through homeassistant mqtt discovery Koenkk/zigbee2mqtt#73

Closed

dgomes mentioned this pull request May 28, 2018

Proposal: Allow changing names through UI for MQTT Discovery home-assistant/architecture#30

Closed

Adminiuga pushed a commit to Adminiuga/home-assistant that referenced this pull request Jun 25, 2018

Add MQTT Sensor unique_id (home-assistant#13318)

c16f38c

* Add MQTT Sensor unique_id * Add test * Update comment

home-assistant locked and limited conversation to collaborators Jul 26, 2018

OttoWinter deleted the mqtt-unique-id branch September 29, 2018 10:32

ghost added integration: mqtt and removed platform: sensor.mqtt labels Mar 21, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add MQTT Sensor unique_id #13318

Add MQTT Sensor unique_id #13318

OttoWinter commented Mar 18, 2018 •

edited

Loading

tinloaf commented Mar 18, 2018

OttoWinter commented Mar 18, 2018

tinloaf commented Mar 18, 2018

OttoWinter commented Mar 18, 2018

balloob commented Mar 21, 2018

balloob commented Mar 21, 2018

OttoWinter commented Mar 21, 2018 •

edited

Loading

balloob commented Mar 21, 2018

thepotoo commented Mar 31, 2018

OttoWinter commented Apr 4, 2018

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

balloob commented Apr 6, 2018

thepotoo commented Apr 6, 2018 via email

fabaff commented Apr 6, 2018 •

edited

Loading

thepotoo commented Apr 6, 2018 via email

OttoWinter commented Apr 6, 2018

thepotoo commented Apr 6, 2018 via email

fabaff commented Apr 6, 2018

OttoWinter commented Apr 6, 2018

balloob commented Apr 7, 2018

OttoWinter commented Apr 7, 2018

balloob commented Apr 8, 2018

Add MQTT Sensor unique_id #13318

Add MQTT Sensor unique_id #13318

Conversation

OttoWinter commented Mar 18, 2018 • edited Loading

Description:

Example entry for configuration.yaml (if applicable):

Checklist:

tinloaf commented Mar 18, 2018

OttoWinter commented Mar 18, 2018

tinloaf commented Mar 18, 2018

OttoWinter commented Mar 18, 2018

balloob commented Mar 21, 2018

balloob commented Mar 21, 2018

OttoWinter commented Mar 21, 2018 • edited Loading

balloob commented Mar 21, 2018

thepotoo commented Mar 31, 2018

OttoWinter commented Apr 4, 2018

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

balloob commented Apr 6, 2018

thepotoo commented Apr 6, 2018 via email

fabaff commented Apr 6, 2018 • edited Loading

thepotoo commented Apr 6, 2018 via email

OttoWinter commented Apr 6, 2018

thepotoo commented Apr 6, 2018 via email

fabaff commented Apr 6, 2018

OttoWinter commented Apr 6, 2018

balloob commented Apr 7, 2018

OttoWinter commented Apr 7, 2018

balloob commented Apr 8, 2018

OttoWinter commented Mar 18, 2018 •

edited

Loading

Example entry for `configuration.yaml` (if applicable):

OttoWinter commented Mar 21, 2018 •

edited

Loading

fabaff commented Apr 6, 2018 •

edited

Loading