Publish future raises AWS_ERROR_MQTT_CONNECTION_DESTROYED #350
Description
Describe the bug
In our design I add a callback to the future returned by the publish method:
publish_future, pkt_id = self._mqtt_connection.publish(topic, pkt[1], mqtt.QoS.AT_LEAST_ONCE, retain=False)
publish_future.add_done_callback(lambda x: self._publish_future_callback(x, pkt, topic))
This normally works fine but I have been testing the code for robustness using a repeating cycle in and out of connected modes and on one occasion I got this exception: AWS_ERROR_MQTT_CONNECTION_DESTROYED
After this the sdk still appears to respond correctly but never publishes any packets. It even indicates it is resuming a connection but I don't think it really is.
I have added a workaround to recreate the connection after I get this exception but I'd like to understand why it occurs.
Expected Behavior
For this exception to not get thrown and if it does for other library methods to then fail instead of appearing to work.
Current Behavior
Occasionally throws AWS_ERROR_MQTT_CONNECTION_DESTROYED and then fails to publish afterwards although the library seems to respond like it is working:
exception calling callback for <Future at 0xffff74059160 state=finished raised AwsCrtError>
Traceback (most recent call last):
File "/bba_app/aws_iot_core.py", line 154, in _publish_future_callback
result = publish_future.result(timeout=0.0)
File "/usr/lib/python3.9/concurrent/futures/_base.py", line 433, in result
return self.__get_result()
File "/usr/lib/python3.9/concurrent/futures/_base.py", line 389, in __get_result
raise self._exception
awscrt.exceptions.AwsCrtError: AWS_ERROR_MQTT_CONNECTION_DESTROYED: Connection has started destroying process, all uncompleted requests will fail.
Reproduction Steps
- Create connection
- Start connection
- Publish every 10 seconds for 5 minutes
- Shutdown connection
- Idle 2 minutes
- Repeat
Possible Solution
Are the AWSCRT objects persistent even with create context calls?
Additional Information/Context
Can someone explain what this message means and what can cause it? If there is an error in how I am using the sdk then perhaps that will make it clear.
SDK version used
1.11.3
Environment details (OS name and version, etc.)
Debian, Python 3.8