#21072 Match endpoint name in the mqtt topic based on endpoints listed in the converter

[grafalex82]

Original issue report: #21072

Problem: Zigbee2mqtt uses predefined endpoint names list when matching mqtt topic name or mqtt message payload. If a device is using a non-standard endpoint name, mqtt messages sent to this endpoint are not recognized properly.

Fix: This change enables matching of mqtt topics and payload without using the predefined endpoints list. Instead it gets a list of endpoint names from the device definition's (converter).

The are 2 changes to make this happen:

• Topic name matching is corrected so that does not use the predefined endpoints list. This involves solving an ambiguouty between device/name/with/slashes and device/endpoint.
• Matching attribute_endpoint-like payload involves collecting the list of device endpoint and matching the endpoint name according to that list.

In addition to make tests happy, the following 3 changes were done:

• device.ts: Not all converters provide the full name-to-id map (for example https://github.com/Koenkk/zigbee-herdsman-converters/blob/master/src/devices/gledopto.ts#L310 provide names only for 2 of 4 endpoints). As result reverse match from ID to endpoint name failed for endpoints without name. Now the null is returned instead.
• publish.test.js: The test tries to get state_center from the device that does not have center endpoint. Previous implementation treated center as endpoint name and displayed Device 'wall_switch_double' has no endpoint 'center'. As soon as center endpoint is not stated on the endpoint list, the new code treats state_center as an attribute name, and therefore the error message changes to No converter available for 'state_center'.
• publish.ts: if(localTarget == null) block was removed - endpoint name shall be matched by the regexp above (which now matches only defined endpoint names), so getting endpoint by name shall always suceed.

[Koenkk]

Looks good!

Could we also get rid of

zigbee2mqtt/lib/util/utils.ts

Line 33 in fdc9f7d

const endpointNames = [

?

[grafalex82]

I think this is possible. But do you think it worth doing that in a single PR?

I did a quick look, and this list is used in 4-5 places across the code. Some are easy to fix, other will require some work, and probably additional changes accross the codebase.

[Koenkk]

I'm fine with doing it in a separate pr. I will release a hotfix z2m release soon and will merge this PR after that.

[grafalex82]

Thanks.

FYI: another PR on getting rid of endpointNames. It is independent of this PR, and can be merged in any order.

[grafalex82]

Since this PR is waiting to be merged anyway, I did a small additional change on the same topic. It was reasonable to make it based on this PR change, rather than separately. Please take a look before merging.

[Koenkk]

Thanks!

[kirovilya]

I discovered that the logic for working with endpoints has changed here.
As a result, some Tuya-devices with several “dummy”-endpoints stopped working.

For example, like here https://github.com/Koenkk/zigbee-herdsman-converters/blob/master/src/devices/tuya.ts#L1745
In this case, states "state_l1", "state_l2", etc. are created in the web-frontend. But, when trying to manage them, an error appears like: No converter available for 'state_l1'.

Reason - the corresponding endpoint names were not found in the list https://github.com/Koenkk/zigbee2mqtt/blob/dev/lib/extension/publish.ts#L199 - for such Tuya-devices, there is one last endpoint ['l4'].
I tried to solve it here #21550, but all the tests are not adapted and diverge.

[grafalex82]

Thanks for reporting, I think I understand the issue.
I'll take a look early next week.