publishQueueDisabledException

[xelfer]

Hi again, I've just tried starting up my code that was working after I fixed my issue in issue 4, but it's now throwing this:

Traceback (most recent call last):
  File "./awstemp.py", line 38, in <module>
    myDeviceShadow.shadowUpdate(tempreading, None, 5)
  File "/usr/local/lib/python2.7/dist-packages/AWSIoTPythonSDK/core/shadow/deviceShadow.py", line 374, in shadowUpdate
    self._shadowManagerHandler.basicShadowPublish(self._shadowName, "update", JSONPayloadWithToken)
  File "/usr/local/lib/python2.7/dist-packages/AWSIoTPythonSDK/core/shadow/shadowManager.py", line 65, in basicShadowPublish
    self._mqttCoreHandler.publish(currentShadowAction.getTopicGeneral(), srcPayload, 0, False)
  File "/usr/local/lib/python2.7/dist-packages/AWSIoTPythonSDK/core/protocol/mqttCore.py", line 350, in publish
    raise publishQueueDisabledException()
core.exception.AWSIoTExceptions.publishQueueDisabledException

Is this something I have done? my certificate is "ACTIVE".. shadow status says it's in sync (although 4 days since an update). I'm not sure what I may have done to break this.

This is in the ap-southeast-2 region.

Thanks,
xelfer

[liuszeng]

Hi @xelfer ,

Thank you very much for using AWS IoT Python SDK shadow functionality!

The AWS IoT Python SDK provides auto-reconnect/resubscribe functionality for both plain MQTT and device shadow. It provides offline publish requests queuing for plain MQTT only since device shadow contains time sensitive requests that may alter the version/content of the device shadow JSON document. See here (Offline Publish Requests Queuing section): https://github.com/aws/aws-iot-device-sdk-python#id4

The offline publish requests queuing functionality is disabled for AWSIoTMQTTShadowClient by default. When the client is offline, the SDK will try to put the outbound publish requests into the internal queue and will raise publishQueueDisabledException if the queue is disabled, which is the case for shadow. You can choose to enable the offline publish requests queuing functionality by changing the configuration of AWSIoTMQTTClient underneath:

MQTTClient = myShadowClient.getMQTTConnection()
MQTTClient.configureOfflinePublishQueueing(yourQueueSize, yourDropBehavior)

Since device shadow contains time sensitive data like real time sensor data that you might not want to queue up and resend, a recommended way is to ignore this exception without changing the underneath AWSIoTMQTTClientso that no shadow request is queued.

Hope that answers your question.

Thanks,
Liusu

[xelfer]

Hi @liuszeng

Thanks again for your very prompt reply.

I'm still trying to understand the issue. Are you suggesting that the queue i'm pushing to is offline? is this on the AWS end? (and is it my fault?).

I don't mind not queuing for resend, it's just a thermometer reading. I'm just wondering why I'm hitting the error at all.

Also, is there a simple way to ignore the exception? and how can I figure out why I can't update my shadow at all?

Thanks,
xelfer

[liuszeng]

Hi @xelfer ,

From the stack trace you are showing, it seems that your client gets connected to AWS IoT and somehow gets disconnected. The SDK then tries to reconnect (with back off). Before it successfully reconnects, the SDK will try to enqueue the shadow requests (MQTT publish requests) into its internal queue (the so called offline publish request queue) instead of directly send them out. As I mentioned in the previous reply, the internal queue is disabled by default for AWSIoTMQTTShadowClient and that's the reason it throws out an exception. You can choose to enable the queue through SDK API or ignore the exception using try.. except...

There is a more detailed example that you can refer to about how you can ignore the disabled queue exception in using IoT Python SDK:
https://github.com/aws/aws-iot-device-sdk-arduino-yun/blob/master/ExampleAppScript/ThermostatSimulatorApp/ThermostatSimulatorApp.py#L40
This example simulates the application side using AWS IoT Python SDK for a thermostat as a controller that updates the desired state of the device shadow, which I think might fit into your use case.

As for checking the result of a shadow update request, you will need to specify a callback for the shadow request (right now you are setting it to be None). Once there is a response from AWS IoT, your callback will be triggered and gives you the information about whether this request has been accepted/rejected/timed out.

Thanks,
Liusu

[xelfer]

Thank you, I'll re-implement the callback function and see if I can figure out what's going on. :)

[xelfer]

Well, I had all kinds of issues trying to reimplement the device shadow callback.. but somewhere during testing my device shadow updated properly. I removed all my changes and tested my script as it was before I started (from what I have in github) and it's working properly.

I can't figure out why, nothing has changed on my end. I'm suspecting there may have been some issue with something on the AWS endpoint? :/ oh well. I'm happy it's working for now at least.

Thanks for your help @liuszeng

[liuszeng]

Hi @xelfer ,

Just want to add a couple of things you can check for similar issues:

A. About device shadow callback
The device shadow functionality in AWS IoT Python SDK provides async API that helps you to start a shadow request and tracks the response through registered callback. The callback that you registered shall have certain function signature in order to be triggered by the SDK. Take shadow update as an example, the API is like this:

shadowUpdate(srcJSONPayload, srcCallback, srcTimeout)

where srcCallback shall have the following function signature:

customCallback(payload, responseStatus, token)

where payload is the service response in forms of JSON document, responseStatus indicates whether the received response is accepted, rejected, time out or delta messages, token is the internal string generated by the SDK to track this request.

If you look at one of the examples provided in the SDK, you should be able to see how the shadow callback is defined and works. Take basicShadow/basicShadowUpdater.py as an example, you can find that there is a response callback defined for shadow update:

def customShadowCallback_Update(payload, responseStatus, token):
    # payload is a JSON string ready to be parsed using json.loads(...)
    # in both Py2.x and Py3.x
    if responseStatus == "timeout":
        print("Update request " + token + " time out!")
    if responseStatus == "accepted":
        payloadDict = json.loads(payload)
        print("~~~~~~~~~~~~~~~~~~~~~~~")
        print("Update request with token: " + token + " accepted!")
        print("property: " + str(payloadDict["state"]["desired"]["property"]))
        print("~~~~~~~~~~~~~~~~~~~~~~~\n\n")
   if responseStatus == "rejected":
        print("Update request " + token + " rejected!")

B. About connect/disconnect
It seems that you are experiencing connect/disconnect issue when you are using the client. One thing you want to make sure is that you are not connecting two clients using the same client Id within the same account. If you do that, the two clients will keep kicking out each other and affect the connection to be unstable.

Please feel free to reopen the issue or start another thread if you have more questions.

Thanks,
Liusu

[xelfer]

@liuszeng It is entirely possible I had two clients kicking each other off. I was moving my sensor from one raspberry pi to another, maybe I didn't stop the first one.

Thank you for your assistance, I've learnt a great deal about how the shadow callback works. 😄

[QuinnCiccoretti]

To confirm, I got this error when I was connecting two clients (one was a greengrass lambda) with the same credentials.