MQTT reconnect is slow (IDFGH-508)

[kanstrup]

MQTT reconnect is slow

Environment

• Development Kit: [none]
• Module or chip used: [ESP32]
• IDF version (run git describe --tags to find it): v3.3-beta1-64-gd8b625a
• Build System: [Make]
• Compiler version (run xtensa-esp32-elf-gcc --version to find it): 1.22.0-80-g6c4433a
• Operating System: [Linux]
• Power Supply: [external 5V]

Problem Description

MQTT takes longer time than necessary to reconnect after network connection has been re-established.

esp-mqtt library is unaware of underlying network connectivity states and current auto reconnect mechanism is built around timed retry attempts every MQTT_RECONNECT_TIMEOUT_MS.

MQTT_RECONNECT_TIMEOUT_MS is set to 10 seconds per default and together with a bug that makes the reconnect delay another MQTT_RECONNECT_TIMEOUT_MS makes MQTT reconnection between 10-20 seconds slower than it has to.

As application code usually keeps track of network connectivity state exporting a new function to force reconnect would help applications that need instant reconnect.

I've opened a pull request on EspressIf/esp-mqtt repo that both export the new reconnect function and solves the above mentioned extra reconnect delay. As it's a submodule I'm not sure you are aware of it.

espressif/esp-mqtt#83

Expected Behavior

1. Connect to wifi network
2. Connect mqtt
3. Wait for wifi disconnect (this disconnect can be any reason)
4. Wait for wifi reconnect
5. Observe mqtt instant reconnect

Actual Behavior

1. Connect to wifi network
2. Connect mqtt
3. Wait for wifi disconnect (this disconnect can be any reason)
4. Wait for wifi reconnect
5. Notice that after wifi successfully reconnected mqtt takes another 10-20 seconds before connecting.

Steps to reproduce

1. Apply patch 0001-MQTT-slow-reconnect-example.patch
2. Build and flash examples/protocols/mqtt/ssl example application.
3. Run for a while and look for these printouts:

I (5570) MQTTS_EXAMPLE: MQTT (re-)connect took 2.00 seconds
I (22360) MQTTS_EXAMPLE: MQTT (re-)connect took 15.00 seconds
I (39170) MQTTS_EXAMPLE: MQTT (re-)connect took 14.00 seconds
...

Notice the initial mqtt connect takes about 2 seconds. Reconnecting takes 10-20 seconds.

Code to reproduce this issue

examples/protocols/mqtt/ssl example application with patch 0001-MQTT-slow-reconnect-example.patch applied.

Solution

1. Pull mqtt reconnect improvements esp-mqtt#83
2. Apply 0002-Force-MQTT-reconnect-on-wifi-connected.patch
3. Build and flash examples/protocols/mqtt/ssl example application
4. Run for a while and look for these printouts:

I (5541) MQTTS_EXAMPLE: MQTT (re-)connect took 2 seconds
I (9541) MQTTS_EXAMPLE: MQTT (re-)connect took 2 seconds
I (13371) MQTTS_EXAMPLE: MQTT (re-)connect took 2 seconds
...

Notice both initial mqtt connect and reconnect takes about 2 seconds.

Other items if possible

0001-MQTT-slow-reconnect-example.patch.txt

0002-Force-MQTT-reconnect-on-wifi-connected.patch.txt