MQTT publish causes memory exhaustion if client not connected (IDFGH-3748) #5668
Comments
github-actions
bot
changed the title
|
Hi , @RazMake, Thanks for reporting this. I agree, we should definitely do something to prevent. I'll look into it. |
david-cermak
pushed a commit
to espressif/esp-mqtt
that referenced
this issue
Aug 31, 2020
As long as the client was disconnect no cleanup were performed, consuming memory for every message published. Even if they were already expired and would be discarded when reconnected. Also moves delete of expired msgs before retransmit is done. This avoids situtations where a message could be sent even if it was expired. Closes espressif/esp-idf#5668
espressif-bot
pushed a commit
that referenced
this issue
Sep 10, 2020
SSL: add config option for skipping common name check esp-mqtt commit: espressif/esp-mqtt@5e8950e) Closes espressif/esp-mqtt#158 SSL: add support for tls with secure element (ATECC608A) esp-mqtt commit: espressif/esp-mqtt@a7ff9af) Closes espressif/esp-mqtt#156 Websocket: Allow the query part of the uri to be a part of the path esp-mqtt commit: espressif/esp-mqtt@40b06de) Closes espressif/esp-mqtt#161 Config: Add check for consistency between config settings esp-mqtt commit: espressif/esp-mqtt@8a412c1) Add IDF version check for secure element feature esp-mqtt commit: espressif/esp-mqtt@db4bce0) Fix esp_mqtt_client_stop deadlock esp-mqtt commit: espressif/esp-mqtt@5e17dca) Closes espressif/esp-mqtt#163 Add dispatch error event for read errors esp-mqtt commit: espressif/esp-mqtt@d4aaec0 Closes #5704 Cleanup expired messages when offline esp-mqtt commit: espressif/esp-mqtt@bdadd77 Closes #5668 esp_mqtt_client_publish now returns msg id for QoS > 0 when offline esp-mqtt commit: espressif/esp-mqtt@f7325bf
espressif-bot
pushed a commit
that referenced
this issue
Sep 22, 2020
SSL: add config option for skipping common name check esp-mqtt commit: espressif/esp-mqtt@5e8950e) Closes espressif/esp-mqtt#158 SSL: add support for tls with secure element (ATECC608A) esp-mqtt commit: espressif/esp-mqtt@a7ff9af) Closes espressif/esp-mqtt#156 Websocket: Allow the query part of the uri to be a part of the path esp-mqtt commit: espressif/esp-mqtt@40b06de) Closes espressif/esp-mqtt#161 Config: Add check for consistency between config settings esp-mqtt commit: espressif/esp-mqtt@8a412c1) Add IDF version check for secure element feature esp-mqtt commit: espressif/esp-mqtt@db4bce0) Fix esp_mqtt_client_stop deadlock esp-mqtt commit: espressif/esp-mqtt@5e17dca) Closes espressif/esp-mqtt#163 Add dispatch error event for read errors esp-mqtt commit: espressif/esp-mqtt@d4aaec0 Closes #5704 Cleanup expired messages when offline esp-mqtt commit: espressif/esp-mqtt@bdadd77 Closes #5668 esp_mqtt_client_publish now returns msg id for QoS > 0 when offline esp-mqtt commit: espressif/esp-mqtt@f7325bf
espressif-bot
pushed a commit
that referenced
this issue
Oct 30, 2020
SSL: add config option for skipping common name check esp-mqtt commit: espressif/esp-mqtt@5e8950e) Closes espressif/esp-mqtt#158 Websocket: Allow the query part of the uri to be a part of the path esp-mqtt commit: espressif/esp-mqtt@40b06de) Closes espressif/esp-mqtt#161 Config: Add check for consistency between config settings esp-mqtt commit: espressif/esp-mqtt@8a412c1) Add IDF version check for secure element feature esp-mqtt commit: espressif/esp-mqtt@db4bce0) Fix esp_mqtt_client_stop deadlock esp-mqtt commit: espressif/esp-mqtt@5e17dca) Closes espressif/esp-mqtt#163 Add dispatch error event for read errors esp-mqtt commit: espressif/esp-mqtt@d4aaec0 Closes #5704 Cleanup expired messages when offline esp-mqtt commit: espressif/esp-mqtt@bdadd77 Closes #5668 esp_mqtt_client_publish now returns msg id for QoS > 0 when offline esp-mqtt commit: espressif/esp-mqtt@f7325bf
espressif-bot
pushed a commit
that referenced
this issue
Nov 10, 2020
SSL: add config option for skipping common name check esp-mqtt commit: espressif/esp-mqtt@5e8950e) Closes espressif/esp-mqtt#158 Websocket: Allow the query part of the uri to be a part of the path esp-mqtt commit: espressif/esp-mqtt@40b06de) Closes espressif/esp-mqtt#161 Config: Add check for consistency between config settings esp-mqtt commit: espressif/esp-mqtt@8a412c1) Add IDF version check for secure element feature esp-mqtt commit: espressif/esp-mqtt@db4bce0) Fix esp_mqtt_client_stop deadlock esp-mqtt commit: espressif/esp-mqtt@5e17dca) Closes espressif/esp-mqtt#163 Add dispatch error event for read errors esp-mqtt commit: espressif/esp-mqtt@d4aaec0 Closes #5704 Cleanup expired messages when offline esp-mqtt commit: espressif/esp-mqtt@bdadd77 Closes #5668 esp_mqtt_client_publish now returns msg id for QoS > 0 when offline esp-mqtt commit: espressif/esp-mqtt@f7325bf
david-cermak
pushed a commit
to espressif/esp-mqtt
that referenced
this issue
Dec 16, 2022
SSL: add config option for skipping common name check esp-mqtt commit: 5e8950e) Closes #158 SSL: add support for tls with secure element (ATECC608A) esp-mqtt commit: a7ff9af) Closes #156 Websocket: Allow the query part of the uri to be a part of the path esp-mqtt commit: 40b06de) Closes #161 Config: Add check for consistency between config settings esp-mqtt commit: 8a412c1) Add IDF version check for secure element feature esp-mqtt commit: db4bce0) Fix esp_mqtt_client_stop deadlock esp-mqtt commit: 5e17dca) Closes #163 Add dispatch error event for read errors esp-mqtt commit: d4aaec0 Closes espressif/esp-idf#5704 Cleanup expired messages when offline esp-mqtt commit: bdadd77 Closes espressif/esp-idf#5668 esp_mqtt_client_publish now returns msg id for QoS > 0 when offline esp-mqtt commit: f7325bf Add support for Digital Signature through ESP-TLS esp-mqtt commit: 7d8e59d
egnor
pushed a commit
to egnor/esp-mqtt
that referenced
this issue
Dec 23, 2022
SSL: add config option for skipping common name check esp-mqtt commit: espressif@5e8950e) Closes espressif#158 SSL: add support for tls with secure element (ATECC608A) esp-mqtt commit: espressif@a7ff9af) Closes espressif#156 Websocket: Allow the query part of the uri to be a part of the path esp-mqtt commit: espressif@40b06de) Closes espressif#161 Config: Add check for consistency between config settings esp-mqtt commit: espressif@8a412c1) Add IDF version check for secure element feature esp-mqtt commit: espressif@db4bce0) Fix esp_mqtt_client_stop deadlock esp-mqtt commit: espressif@5e17dca) Closes espressif#163 Add dispatch error event for read errors esp-mqtt commit: espressif@d4aaec0 Closes espressif/esp-idf#5704 Cleanup expired messages when offline esp-mqtt commit: espressif@bdadd77 Closes espressif/esp-idf#5668 esp_mqtt_client_publish now returns msg id for QoS > 0 when offline esp-mqtt commit: espressif@f7325bf Add support for Digital Signature through ESP-TLS esp-mqtt commit: espressif@7d8e59d
egnor
pushed a commit
to egnor/esp-mqtt
that referenced
this issue
Dec 23, 2022
SSL: add config option for skipping common name check esp-mqtt commit: espressif@5e8950e) Closes espressif#158 SSL: add support for tls with secure element (ATECC608A) esp-mqtt commit: espressif@a7ff9af) Closes espressif#156 Websocket: Allow the query part of the uri to be a part of the path esp-mqtt commit: espressif@40b06de) Closes espressif#161 Config: Add check for consistency between config settings esp-mqtt commit: espressif@8a412c1) Add IDF version check for secure element feature esp-mqtt commit: espressif@db4bce0) Fix esp_mqtt_client_stop deadlock esp-mqtt commit: espressif@5e17dca) Closes espressif#163 Add dispatch error event for read errors esp-mqtt commit: espressif@d4aaec0 Closes espressif/esp-idf#5704 Cleanup expired messages when offline esp-mqtt commit: espressif@bdadd77 Closes espressif/esp-idf#5668 esp_mqtt_client_publish now returns msg id for QoS > 0 when offline esp-mqtt commit: espressif@f7325bf Add support for Digital Signature through ESP-TLS esp-mqtt commit: espressif@7d8e59d
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
ESP-IDF Version: 4.0.1The code is fairly simple, publish once a minute a (temperature) sensor value to MQTT, with QoS=1.
If the client disconnects (server unavailable, connectivity issues) for some time it looks like outbox grows until it runs out of memory causing problems for the rest of the components. Upon reconnect it looks like the outbox is trimmed, dropping messages older than 30 secs (per mqtt_client.c), freeing all that memory (if it doesn't cause a CoreDump before due to memory exhaustion). Probably the same outbox trimming that happens in connect should happen from time to time in publish/enqueue as well. In my empirical tests I was running out of memory in about 7 hours (obviously varies with payload length, frequency, what other memory is consumed).
Also, maybe offer a configuration value to limit outbox usage to no more than X% of memory. Or a number of entries, something that does not allow it to grow unbounded.
The text was updated successfully, but these errors were encountered: