Reconnecting smoothly

[mbaroody]

aws-iot-device-sdk-python/AWSIoTPythonSDK/core/protocol/mqtt_core.py

Line 248 in 47363e9

raise publishTimeoutException()

Hello,

First thanks for creating/maintaining this sdk. From the documentation, it states that you provide:

It provides basic synchronous MQTT operations in the classic MQTT publish-subscribe model, along with configurations of on-top features:
- Auto reconnect/resubscribe
- Progressive reconnect backoff
- Offline publish requests queueing with draining

However, I'm not experiencing these features. I expect when I call awsMQTTClient.publish(topic, message, qos) I expect not to ever get a publishTimeoutException(), except maybe when the maxReconnectQuietTimeSecond in configureAutoReconnectBackoffTime (docs) is exceeded. However, I get publishTimeoutException() quite often.

Do I have to do something like this now everytime I publish?

        try:
            myMQTTClient.publish("testing/IoT", message, 1)
        except:
            print(err)
            myMQTTClient.connect()
            myMQTTClient.publish("testing/IoT", message, 1)

What's the proper way of recovering from this type of timeout?

[hedeshianaws]

Thank you for using the AWS IoT Python Device SDK. Are you configuring the message queueing and auto-reconnect parameters? If so, what values are you using?

You can see a sample configuration here:

aws-iot-device-sdk-python/samples/basicPubSub/basicPubSub.py

Lines 91 to 93 in 47363e9

	# AWSIoTMQTTClient connection configuration
	myAWSIoTMQTTClient.configureAutoReconnectBackoffTime(1, 32, 20)
	myAWSIoTMQTTClient.configureOfflinePublishQueueing(-1) # Infinite offline Publish queueing

[mbaroody]

These are the options we are using:

    mqtt = AWSIoTMQTTClient(client)
    mqtt.configureEndpoint(endpoint, 8883)
    mqtt.configureCredentials(rootCA, privKey, cert)
    mqtt.configureOfflinePublishQueueing(-1)
    mqtt.configureDrainingFrequency(2)
    mqtt.configureConnectDisconnectTimeout(10)
    mqtt.configureMQTTOperationTimeout(5)
    mqtt.configureAutoReconnectBackoffTime(1, 128, 20)

We also have at least two concurrent threads calling mqtt.publish... Are there any known problems with the mqtt client and threading?

[liuszeng]

Hi @mbaroody ,

There are several thing I need to point out here:

1. publishTimeoutException is raised any time when the client is performing QoS1 synchronous publish and is not able to receive the PUBACK from the server within the configured timeout. This is normal and could happen due to many reasons, e.g., unstable network connectivity.

2. SDK will handle the auto-reconnect for you in the background after the underneath Paho has detected the offline event. You do not need to invoke connect by yourself. How fast the underneath Paho can detect the offline event depends on the keep-alive configuration. If you want that to be detected faster, you need to configure the keep-alive to be shorter so that the SDK is pinging the server faster.

As of now, we did not receive reports from our customer regarding threading issues with the SDK. I have the following questions:

1. Are you invoking publish in the main thread or, by any chance, in the callback?
2. Do you have any log output from the SDK when the issue is happening?

Thanks,
Liusu

[liuszeng]

Closing due to inactivity.

[GlennS]

Just noting that I had a similar issue when running inside uwsgi (using Flask), in case anyone else comes across this and needs help.

I fixed it by making sure to construct and connect my AWSIoTMQTTClient instance in the same context (probably a thread?) that requests run in. I still reuse the instance between different requests though, because calling connect() on the client takes some time.

[deeshank]

Just noting that I had a similar issue when running inside uwsgi (using Flask), in case anyone else comes across this and needs help.

I fixed it by making sure to construct and connect my AWSIoTMQTTClient instance in the same context (probably a thread?) that requests run in. I still reuse the instance between different requests though, because calling connect() on the client takes some time.

Hi GlennS,

I am using Flask and trying to publish a message via MQTT but I end up getting into on_offline callback and i am unable to publish message too. If you can let me know how you got it working, that would be really helpful.

[prerna2808]

I am having a similar issue. Publish timed out.
After any uncertain period of time.
Any help would be appreciated.
Thankyou

[alonsnir]

@liuszeng

SDK will handle the auto-reconnect for you in the background after the underneath Paho has detected the offline event. You do not need to invoke connect by yourself. How fast the underneath Paho can detect the offline event depends on the keep-alive configuration. If you want that to be detected faster, you need to configure the keep-alive to be shorter so that the SDK is pinging the server faster.

If publish fails due to connectivity issue, will it invoke the reconnect or will it only happen on the next keep alive period?
Is there a difference in this behaviour between QoS0/1?

[graebm]

Publishes do not affect the client's attempts to reconnect. Reconnects occur when the underlying socket reports that the connection is lost, or when the MQTT Client doesn't get a response to the PING packets it is sending to the server at regular intervals ("keep alive" values affect the rate that these PING packets are sent)