How to receive and store all published messages if a client goes offline?

[saba-sabrin]

Hello! I am working with MQTT Broker and Client using MQTTnet library. Using the latest version. I have seen a situation where if a client goes offline after subscribing to a topic then, all the messages published on that during that time is not shown when the client reconnects. I have used the following for the client configuration:

Clean Session = false
QoS = 1 or 2 (AtMostOnce or ExactlyOnce)

According to my understanding, the server should have an internal queue where it should store the messages and when client reconnects, there should be a possibility to deliver all the messages received when the client was offline. I do understand the message retain policy. I have checked the json file being generated by the server where it received only the last message on the topic and when the retain flag is true, the Client when reconnects, only gets the single last message published on that subscribed topic.

My question is, how can I receive all the messages which was published when the client was offline. Please help me on this.
Thank You

[saba-sabrin]

Any help on this topic would be greatly appreciated !! Any kind of suggestions are welcome! Thanks.

[fogzot]

If you are interested in the last message for every topic, just set the retain flag when publishing the message. The server will store the last retained message and send it to the client when the client (re)connects and subscribes to that topic (works with wildcards too).

If you want the full set of all messages received by server while the client was disconnected, then MQTT doesn't help. You need to provide your own logic to save the messages and "replay" them when the client reconnect. MQTTnet provides all the plumbing you need but not an out-of-the-box solution for this case. This part isn't true - just forgot about persistent sessions.

[chkr1011]

MQTTnet also has such a queue but you use CleanSession=true. This will delete the session and thus the pending messages of that session.

[saba-sabrin]

@chkr1011

I am already using "Clean Session" as false so that, the session is saved. But, whenever my client reconnects, it doesn't get the published messages while it was offline.

[chkr1011]

Please try to set the EnablePersistentSessions property to true at the server options.

[saba-sabrin]

"WithPersistentSession" property had worked for my Asp.Net Core 2.0 App! Thanks a lot!

[JanEggers]

@saba-sabrin I think we just store the messages in memory, so if the broker is restarted you will still loose messages.

@chkr1011 should we extend IMqttServerStorage or provide a similar interface like IMqttSessionStorage to store pending messages of a session?

[chkr1011]

@JanEggers Yes that is a good idea.

[saba-sabrin]

@JanEggers Yes, you are absolutely right. I was planning to implement my own interface to store the undelivered messages if the broker restart or break due to some reason. But, if this can be a part of the library, that will be awesome! Thanks!

[M144-Coder]

@JanEggers @chkr1011 Any updates (or a guide) on saving session to storage?

[chkr1011]

This feature is still in progress. I already added some internal interfaces and objects already but the feature is not finished. The problem is not how to load the data when the service starts. It is the saving of that information. My conclusion after some tries for implementation is that this is fully up to the user to save whenever needed. So he needs to use a timer or save on every incoming or queued message or on server shutdown or whatever. I am not able to provide one interface which just needs to be implemented. I hope to release the loading part soon (in a couple of month) so that you can at least load persisted sessions.

[M144-Coder]

Thank you for your reply. I think this feature should have an interface with four methods:
1.To save any message which it's delivery failed.
2.To load messages for a connected client.
3.To save messages on server shutdown by leveraging the HostedService.
4.To load messages when server starts by leveraging the HostedService.

Then it's up to the user to implement the interface and save the messages.

[chkr1011]

The thing is that this is not only covering messages. The MQTT spec says that also some more data must be stored like open QoS level 2 transactions etc. That is the problem I am trying to solve.

And you are missing some use cases. You only covering failed messages. Other people might want to save queued messages before delivering. That can be done on shutdown but that might not happen if the app is crashing.

[JCEdward]

The thing is that this is not only covering messages. The MQTT spec says that also some more data must be stored like open QoS level 2 transactions etc. That is the problem I am trying to solve.

And you are missing some use cases. You only covering failed messages. Other people might want to save queued messages before delivering. That can be done on shutdown but that might not happen if the app is crashing.

Now, Any updates (or a guide) on saving session to storage?

[joergmetzler]

Any updates for this feature?
I would love to test the session persistence :)

[zaksnet]

+1

[onurcanyilmaz]

Any updates for this feature?

[chkr1011]

No new updates yet. Some more internal preparations are made but no public API.

[Nick-BH]

+1