Support upload of files larger than 32Mb #473

Closed
andrerod opened this Issue Nov 12, 2012 · 3 comments

Comments

Projects
None yet
2 participants

It is currently not possible to upload a file larger than 32Mb with blob storage SDK.

In lib/utils/constants.js, max upload size is 32 MB.

DEFAULT_SINGLE_BLOB_PUT_THRESHOLD_IN_BYTES: 32 * 1024 * 1024. This is the value used by file upload code in

lib/services/blob/blobservice.js

Note: Azure Rest API defines max size as 64 MB for a single upload.

From http://msdn.microsoft.com/en-us/library/windowsazure/dd179451.aspx
The maximum upload size for a block blob is 64 MB. If your blob is larger than 64 MB, you must upload it as a set of blocks. For more information, see the Put Block (REST API) and Put Block List (REST API) operations. It's not necessary to call Put Blob if you upload the blob as a set of blocks.

For larger file uploads, API should use Block Lists. Block List support is missing from SDK. Streaming uploads to Azure Blob storage will require use of Block Lists as length of stream in not known before starting uploads. Upload stream will need to be chunked into 4MB blocks and uploaded using Put Block.

We have this support in one of our samples. We decided at the time to not make it part of the SDK Service layer. That'll be part of the convinience layer. But check the samples for examples on how to build this.

As for the 32MB vs 64MB we should investigate.

@andrerod which sample?

Duplicate of #477.

andrerod closed this Feb 18, 2013

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment