Data Protocol: Topic

[BenediktBurger]

Question

For the data protocol (via zmq PUB/SUB, see #3), message handling is based on the "topic", the first frame of a published message.

What should the topic be?

Background information on the workings of ZMQ

Be S a subscribers and P the publisher Component:

• If S subscribes to "xy", then P sends any message whose topic starts with "xy", then zmq will send that message to S, even if the topic is "xyz"
• This is done for all subscribers of P (the PUB port does these checks and sends TCP messages accordingly)
• The subscriber will receive the whole message, that is the first frame (topic) and any other possible frames.

Consequences for the topic

• If the topics have variable length, you cannot subscribe to "xy" without (implicitly) getting messages with topic "xyz" as well.
• As the subscriber receives the topic, it can filter messages with the wrong topic ("xyz" instead of "xy") and discard them. These messages have been passed through the network though.
• You can subscribe to all messages by subscribing to ""

Ideas

Regarding topic overlap

What to do about "xyz" being sent to a "xy" subscriber?

• We could add a "topic delimiter", e.g. newline "\n". That way "xy\n" subscribes only to "xy\n" and not to "xyz\n". You still receive both, if you just subscribe to "xy" (because "xy\n" and "xyz\n" start with "xy").
• We don't care about potentially more messages sent and require message filtering

How to define the topic?

Option 1: Topic is Full name of the component

• Con: In this case, the component has to be connected to the control protocol as well (in order to get the namspace
• Pro: Topics are unique in the network
• Pro: You can filter for a namespace (by susbsciribing to the namespace)
• Con: you have to know the current namespace in order to be able to subscribe to devices. Not a big problem though

Addon subtopic: How to mark different messages fo the same Component?
We could allow subtopics by adding the name of the subtopic after the Fullname. If ":" is the separator between full name and subtopic, a topic would look like "Namespace.Component1:voltages".
That allows to subscribe to a part of the messages of Component1 (those regarding voltages) or to subscribe to all messages by subscribint to "Namespace.Component1".
The separator could be the same one talked about in "topic overlap".

Option 2: Not tied to Full Name

• Con: You have a second naming scheme in your network
• Pro: The data protocol works independently from the control protocol
• Pro: More flexibility (one Component can send different things under different topics

My opinion

I think we should go for the full name as the topic.

I like the subtopic part, but we can add it later as well (as it is appended, it is backwards compatible)

If we allow subtopics, we could make the separator obligatory, such that we prevent topic overlap as a bonus.