Need Support for Resuming File Uploads to Blob Storage

[MichaelDenczek]

• OS and version used: Windows 7 x64
• SDK version used: Latest IoT SDK for C

Description of the issue:

No support for resuming paused file uploads to Blob Storage. Especially important for large files.

Code sample exhibiting the issue:

Code based on Blob_UploadFromSasUri, in blob.c.

Console log of the issue:

HTTP status from storage does not indicate success (400)

Which occurs when attempting to resume file uploads.

[ewertons]

Hi @MichaelDenczek ,
the SDK does not provide a way for pausing an upload. How are you pausing it on the first place?

[MichaelDenczek]

Consider the case where a large file is being uploaded to blob storage from an embedded device and the network connectivity is lost. Upon resumption of network connectivity the current support in the IoT SDK requires the file to be re-uploaded in entirety.

[ewertons]

Sounds like a reasonable scenario [emoji suppressed for more formal response].
Let me check the design out and see what we can do to mitigate that.

[GeraldSchmitt]

We have similar needs. We would also like to be able to resume after a IoTHubClient_UploadMultipleBlocksToBlobAsync.
I suggest an API in issue #371.

[GeraldSchmitt]

Hi guys,

About this feature:

• Is this feature somewhere in your backlog?
  • If yes, is there an ETA?
  • If no, would you consider accepting a PR?
    • Would the API described in issue Add the Resume Upload Multiple Blocks To Blob functionnality #371 be OK for the PR?

[tameraw]

@GeraldSchmitt - Currently we do not have plans to add this feature at the moment in the SDK. If you are willing to provide a PR we can review and comment. Most probably would not be able to merge this in before end of June.

[GeraldSchmitt]

Hi @tameraw

I would gladly start to work on a PR, but in order to avoid unnecessary work, wouldn't it be better to first settle down an API everybody agrees on?

I suggest an API in issue #371, could you have a look?

Also, in issue #375, after some discussion, emerged the idea of providing directly the blob URI (this simple solution would solve most of our needs concerning blobs), but you guys seems to need more time to think this through. Did you make any progress?

[GeraldSchmitt]

Hi guys,

I'm leaving Schneider Electric, and I will no longer follow these topics.
@lmussier (Ludovic Mussier) and @alexmrtn (Alexis Martin) will take care of that from now on.
Have fun !

Cheers,

Gérald

[rbagree]

Hi Guys,
Any update on the enhancement in the discussion?

[jspaith]

This is still on our radar but we don't really have an implementation timeline.

[yzhong94]

@rbagree We have multiple conversations with the Azure Storage team to have a Azure Storage C SDK. Can you help confirm if resume is still a requirement from your end?

[rbagree]

@yzhong94 Yes. This is still a requirement as this will allow us to use the SDK in a remote locations where network connectivity is very intermittent.

[lmussier]

@yzhong94 We are on the same use case on our side. So this is still a waited feature on our side.

[yzhong94]

@rbagree @lmussier thank you for confirming! We have not dropped this request and we are working with the blob storage team on a solution that works for future blob storage improvement. We will get back to you on the plan and ETA as soon as it's finalized.

[mms-]

Would be nice to have this. Given that the SDK through MQTT/IoT Hub uses Azure blob upload as the only way to upload files from devices?

[gregman-msft]

Thank you everyone for the feedback on this issue. This will require new work on the blob storage support, which we are transitioning to a new repository. I have opened up a request on your behalf in this new repository here. Please track the request there and feel free to add any additional comments.

[az-iot-builder-01]

@ewertons, @MichaelDenczek, @rbagree, @jspaith, @lmussier, @mms-, @gregman-msft, thank you for your contribution to our open-sourced project! Please help us improve by filling out this 2-minute customer satisfaction survey