[BUG] An MQTT OTA request made by the library to AWS using CBOR encoding results in the Invalid response on the /rejected topic #490
Comments
arkhipenko
changed the title
arkhipenko
changed the title
|
Thank you for bringing this to our attention! We will be looking further into this issue as soon as we can, thank you for your patience |
|
@gmtt - thank you - do you have an estimate of when we can expect some answer/resolution? We are in the middle of implementation and were hoping that the AWS library will work with the AWS backend "as-is". This is quite unexpected honestly. |
|
Hi @arkhipenko, the encoded hex appears to be correct. That makes us suspect that the issue may lie in the cloud part. Could you please enable debug logging to obtain more information about this? Additionally, could you try using the tinycbor version that we tested here: https://github.com/aws/ota-for-aws-iot-embedded-sdk/tree/main/source/dependency/3rdparty instead of the latest version? As we haven't tested the latest version of this library, it's possible that an incompatible library is causing the problem. |
|
Hi @gmtt Meanwhile: |
|
Hi @gmtt |
|
Hi @arkhipenko |
|
Hi @arkhipenko |
|
Thank you @gmtt |
|
Hello @arkhipenko, Firstly, apologies that it is taking a bit longer than expected. I tried with the older release version of OTA library and my firmware upgrade worked as expected. We are working on figuring out what is causing the issue you are seeing. Thank you, |
|
Hi @AniruddhaKanhere - thank you for getting back to me.
Could you please let me know which version of the library did work? I will try that as well, |
|
I tried compiling with releases 2.0.0 and 3.0.0, but there are too many binary API mismatches - it would take a long time to adjust. |
|
Hey @arkhipenko, I was trying the projects in one of our reference integration projects in this repository: https://github.com/FreeRTOS/iot-reference-nxp-rt1060 It uses the OTA lib submoduled here: https://github.com/FreeRTOS/iot-reference-nxp-rt1060/tree/main/Middleware/AWS. The version in use is v3.4.0. One of my colleagues also mentioned that it worked for them when they tried it on Tuesday (13th June). That being said, we are looking into the cloud side of things as well to make sure that there is no breaking change there. Thank you for your patience. |
|
@AniruddhaKanhere - could you please confirm that your colleague tested MQTT-based OTA, and not HTTPS-based (which works fine for me as well)? I struggle to understand how the same code base produces different results - I do not modify the library code in any way! I would also very much like to see the CBOR message sent to the cloud from that test on June 13th, and the response from the cloud. Would your colleague have those by any chance? |
|
@AniruddhaKanhere - just tried the exact version that you mentioned with the same result: followed by the same |
|
Hey @arkhipenko, Yes, my colleague did test MQTT based OTA. And it succeeded. However, the following is the CBOR that his OTA sent (just maybe a configuration issue). Not sure why are there so many That being said, we also sent a valid CBOR to the IoT core and it was rejected. That is what is causing so much confusion to me. I am not sure how one message succeeds and the other doesn't. We are looking into the cloud side as well. Again, thank you for your patience Anatoli as we work on finding the root cause. Cheers, |
|
@AniruddhaKanhere - it is quite interesting. a 20-byte 'b' field would suggest a file 655360 or so bytes long. |
Yes, that is what the bitmap means. But I don't think that the size of the file should matter because we are still requesting the file and while doing that, we do not know the size of the new file. For all it matters, the new file can be double the size and may not fit in the bitmap that we sent at all. I am not sure though so don't quote me on this :). But, yes, the image is quite big as you predicted: That being said, the bitmap has a By the way, we were able to figure out that there is nothing wrong with the cloud side of things. We were just sending wrong CBOR. But this is good news because it now pinpoints that there might be something wrong on the application side of things. And we can figure out what is wrong in your application. :) Thanks, |
|
When you say "we are sending the wrong CBOR", what does it mean exactly? |
|
Ah yes, let me clarify. If we were supposed to send This happened because we assumed that the test setup we were using was aware of the hexadecimal numbers. So when we sent But, when we did send the valid CBOR message, it was accepted by the cloud. Following up on your issue though; would it be possible for you to upload your code (only the parts we need to understand this issue) to github and share with us? That way we can replicate your results. |
|
@AniruddhaKanhere Can you actually pinpoint what is wrong with my CBOR message?: |
|
@AniruddhaKanhere - would it be possible for me to add you to a private Git repo for code sharing? |
|
Hey @arkhipenko, I am not saying that anything is wrong with your CBOR as it is, maybe we are missing something. Can you try the steps mentioned here in this repository: https://github.com/FreeRTOS/iot-reference-esp32c3? I suggest using everything new for this test (including the 'thing' and corresponding certs/keys and policies). All the process should be outlined in the above linked GSG.
I think it should be fine if you think there is nothing proprietary in there which I should not see. |
|
@AniruddhaKanhere - after careful re-examination of my code I found a nasty little bug in the way outgoing MQTT messages were handled corrupting the CBOR message at the very last step of the process. Corrected and I seem to be getting the MQTT file payloads as expected now. |
|
It is great news that you found the bug @arkhipenko! Thanks again for informing us about the issue and the fix, |
|
The original MQTT code assumed payload was always an ASCII string, so the very final buffer copy was done with |
Describe the bug
An MQTT OTA request made by the library to AWS usinf CBOR encoding results in the Invalid response on the /rejected topic
Host
To Reproduce
Expected behavior
I was expecting to start receiving OTA file chunks on MQTT topics.
Additional context
Using the latest
tinycborversion 0.6.0 from intel.Initial request is made successfully
AWS IOT responds with the job document as expected and a few ota methods are invoked as expected:
then a request for the first chunk is published:
this message decoded is:
{"c": "rdy", "f": 0, "l": 4096, "o": 0, "b": h'FFFFFFFF', "n": 1}according to this: https://cbor.me/
the response is:
This is consistent with the failure I reported for the MQTT download agent library here:
aws-samples/aws-iot-mqtt-download-agent#3
Can someone please look into this? The CBOR message seems to produce the bitmap that is rejected by the backend!
The text was updated successfully, but these errors were encountered: