Zigbee2mqtt subscribing inefficient?

[svrooij]

Bug Report

What happened

I have a different mqtt server then most people use. I use EMQX one of the reasons is that is has a nice visual dashboard to display all sorts of information about the mqtt clients that are connected.

This is a small portion of all the subscriptions zigbee2mqtt creates. It currently has 199 different subscriptions.

What did you expect to happen

I would expect it to use the # wildcard to tell the mqtt server that it wants all topics with this multi-level wildcard.
A single zigbee2mqtt/#/get could replace all these subscriptions

• zigbee2mqtt/+/get
• zigbee2mqtt/+/+/get
• zigbee2mqtt/+/+/+/get
• zigbee2mqtt/+/+/+/+/get

And maybe it should just subscribe to zigbee2mqtt/# but than it will also receive it's own messages.

How to reproduce it (minimal and precise)

1. Start zigbee2mqtt

[Koenkk]

Fixed in the latest dev branch, please confirm 😄

[sjorge]

@Koenkk 82080e3 broke groups and devices with a / in them again.

[sjorge]

Even worse, this breaks everything for me!
Even zigbee2mqtt/0x00158d0001ffaffc/get fails, which should definitely work

[svrooij]

Shouldn't devices and groups name's not just deny / in the name?
Or at least in the mqtt topics? It is recommended to only use A-Za-z0-9 and -_.

[sjorge]

I and a few other people use / in the name to provide some order e.g. <floor>/<room>/<device>, this also makes it easy to get only messages for a certain floor/room.

There was some discussion about this a while back as it was broken in a few odd places but they were fix, because suddenly disallowing / would probably break a lot of things for people.

I guess I can spend a few weekend redoing everything to not have / but I am certainly not the only one using this... also since zigbee2mqtt/0x00158d0001ffaffc/get is also broken, I suspect maybe my broker does not support # when used in the middle.

From some testing it is fine if # is the last bit of the subscripting topic only.

zigbee2mqtt/# seems to work fine while zigbee2mqtt/#/get for example doesn't seem to get any message I send it either. I tried with both aedes and mosce.

[sjorge]

It looks to indeed be that none of the brokers I tried support # in the middle of a subscription topic. Aedes is the one I mainly use and I made it print every client that tried to subscribe, all z2m's subcriptions with # was not at the end were returning an empty subscription list!

@Koenkk will we start enforcing certain brokers from now on?

[svrooij]

Sorry @Koenkk for causing trouble.

How about using zigbee2mqtt/# as a single subscription? That should allow all possible options.

[sjorge]

@svrooij eh it happens :) we're all trying to make z2m better and your observation is correct that it does add a lot of subs...

Multi Level: #

The multi-level wildcard covers many topic levels.
The hash symbol represents the multi-level wild card in the topic. 
For the broker to determine which topics match, 
the multi-level wildcard must be placed as the 
last character in the topic and preceded by a forward slash.

-- https://www.hivemq.com/blog/mqtt-essentials-part-5-mqtt-topics-best-practices/

[sjorge]

So looks like both Aedes and Mosce are doing the correct thing here and rejecting those subscriptions.

[svrooij]

BTW. I haven't tested a subscription with something after the # character for EMQX. So if it's not allowed in the specs that should be followed.

[Koenkk]

Reverted and only added it to the cases were it can be applied (should save 38 subscriptions).

Regarding: #3102 (comment) in that case Zigbee2mqtt will also start listening to it's own message which I don't think is a good idea. If zigbee2mqtt would e.g. publish to zigbee2mqtt/my_device it would receive this message again leading to unnecessary messages send to zigbee2mqtt (= traffic).

[sjorge]

Looks, good... everything is working again.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.