allow . in mqtt topics

[telemac]

Feature Request

• Allow dots as topic separator in mqtt.
• Eventually, use / as topic separator in NATS and MQTT, common topic structure could be convenient for applications using both NATS and MQTT protocols.

Use Case:

Allow nats mqtt server to be used with spark plug B :

Sparkplug B specifications

Sparkplug uses spBv1.0 as topic prefix.

Proposed Change:

• add a parameter to allow dots in mqtt topics, eventually URLEncode the topic for the conversion to the NATS topic

Who Benefits From The Change(s)?

• Users that would like to une NATS with Sparkplug B protocol
• Users that would like tu use nats in replacement of other mqtt workers, already having a working system that uses dots in the mqtt topics
• Myself :)

Alternative Approaches

Allow the use of / as separator in NATS, to harmonize the NATS and MQTT topics.

[kozlovic]

@telemac To be sure I understand what you are saying:

add a parameter to allow dots in mqtt topics, eventually URLEncode the topic for the conversion to the NATS topic

If an inbound MQTT topic has a dot, say foo.bar/baz, in MQTT world this is a two "level" topic, with first token being "foo.bar" and second "baz". If this were to be URL encoded, it would become Zm9vLmJhci9iYXo=. How are you going to have NATS applications receive that message? Seem really weird to me.

Whatever character that is used to replace the mqtt topic's '.' with may cause an issue when going from NATS to MQTT. For instance suppose we select the character "_" and assume that when going from MQTT to NATS we convert '.' with '.', then in example above it would become the NATS subject foo_bar/baz. But then any NATS message that had this character as part of the subject would be possibly incorrectly translated to . in MQTT world. And also what if the selected character (let's say _) was originally present in the MQTT topic, such as foo_bar.bat/baz? Then if it becomes foo_bar_bat.baz, then on the way back to MQTT it would become foo.bar.bat/baz, which then your apps would not be able to receive.

I don't see a solution that would work in both directions and be useable for NATS to MQTT (and vice versa) communication. I am opened to more suggestions, but there is a reason I originally did not support the . and other NATS wildcards in MQTT subjects.

[telemac]

Hi @kozlovic

I was thinking of this kind of encoding : https://en.wikipedia.org/wiki/Percent-encoding

So, the mqtt topic foo.bar/baz will become foo%2Ebar/baz in nats

The problematic characters are encoded as %xx where xx is the ascii hex value.

. is encoded as %2E
/ is encoded as %2F
% is encoded as %25

It seems to me that this cloud solve the character collision problem between nats and mqtt.

[NicholasTNG]

Sorry if that is a stupid question, but wouldn't it be possible to just replace . by / and / by .? This doesn't has the problem of the _ in the example above since this is a bijective replacement, so there wouldn't be the danger of an unreplaced / since all of them would be replaced.

EDIT: Ah, it seems like neither of the proposals is backwards compatible (in my case since /// is a valid mqtt-topic that is translated to /././)

[NicholasTNG]

RabbitMQ has the same problem and solves it with a specific sparkplug config option: https://www.rabbitmq.com/mqtt.html#sparkplug-support

[sandrokeil]

Is it possible to support to configure the subject/topic delimiter (.) like in CSV files, so it would be possible to use Sparkplug specification and use a / as delimiter / for hierarchy instead of .? The Sparkplug specification contains a dot in the subject/topic e.g. spBv1.0/plant1/message_type/rolling_mill_1/device1 and uses / for the hierarchy.

So just a possibility to switch between MQTT and NATS separators would be awesome.

[derekcollison]

Yes we are considering for the 2.10 release.

[neilalexander]

This will now be supported in 2.10. Thanks for raising this!