Implement message expiry intervals functionality

[andsel]

General description

Messages crafted for offline devices are stored on the broker, waiting for the device to regain connectivity for delivery. In grand-scale deployments, where the count of connected devices escalates into thousands or even millions, it’s imperative to constrain the offline message queue for each client individually.

The longevity of message relevance to IoT devices can exhibit significant fluctuations. Take the example of the connected car: traffic updates maintain their relevance only briefly. Yet, when we consider over-the-air firmware upgrades, these need to be executed even if the car remains offline for an extended period, stretching into several weeks.

By assigning an optimal message expiry interval to messages with a limited relevance window and leaving perpetual relevance messages without an expiry, we ensure effective utilization of broker resources for clients that could be offline for a considerable duration. This strategy also spares clients from being inundated with irrelevant messages upon reconnection.

References:

• https://www.hivemq.com/blog/mqtt5-essentials-part4-session-and-message-expiry/

MQTT 5 specification (relevant to server implementation)

Publish properties

If present, the Four Byte value is the lifetime of the Application Message in seconds. If the Message Expiry Interval has passed and the Server has not managed to start onward delivery to a matching subscriber, then it MUST delete the copy of the message for that subscriber [MQTT-3.3.2-5].

If absent, the Application Message does not expire.

The PUBLISH packet sent to a Client by the Server MUST contain a Message Expiry Interval set to the received value minus the time that the Application Message has been waiting in the Server [MQTT-3.3.2-6]

[MQTT-4.3.3-13] In the QoS 2 delivery protocol, the receiver MUST continue the QoS 2 acknowledgement sequence even if it has applied Application Message expiry.

Task list

• update retained storage to save also the message expiry an clean the message if it expires. Updates retained messages management to hanbdle message expiry interval #819
• messages laying in a session queue waiting for the client, can't be forwarded to the attached client if the expiry message is elapsed. Feature/message expiry for queued #822
• a forwarded message has to contain the updated message expiry (subtract the time passed into the broker). Update message expiry remaining time and drop queued messages if expired #823
• fix the will publishing to include also message expiry interval when needed. Updated will publishing to contains also the message expire property #824

[andsel]

Close as the task list is completed