BlockBlobService size limits are outdated

[crepererum]

Which service(blob, file, queue) does this issue concern?

blob

What problem was encountered?

These constants are too low for the current version (which is 2017-07-29)

Have you found a mitigation/solution?

The constants should be changed to the following:

• MAX_SINGLE_PUT_SIZE = 256 * 1024 *1024 (ref)
• MAX_BLOCK_SIZE = 100 * 1024 * 1024 (ref)
• MIN_LARGE_BLOCK_UPLOAD_THRESHOLD = 100 * 1024 * 1024 + 1 to match MAX_BLOCK_SIZE

[zezha-msft]

Hi @crepererum, thanks for reaching out!

It is true that these default values can be set higher, but smaller blobs may be more common than larger ones, so a block size of 100MB and single put size of 256MB might lead to worse performance for a lot of cases. In general, it is preferred to let the users of the SDK adjust these numbers to whatever that makes the most sense for their file sizes and network conditions.

Thank you for the suggestion!

[smarlowucf]

In general, it is preferred to let the users of the SDK adjust these numbers to whatever that makes the most sense for their file sizes and network conditions.

I have this problem as well, however there's no way to adjust these numbers. Attempting to upload a page blob instead of a block blob but both have the same problem. In that the max block/page size is not configurable.

https://github.com/Azure/azure-storage-python/blob/master/azure-storage-blob/azure/storage/blob/pageblobservice.py#L1003

Is this an option that can be added to the create_blob_from_stream and create_blob_from_path methods?

[zezha-msft]

Hi @smarlowucf, could you please clarify why you couldn't adjust these numbers?

Please see an example in our tests.

[smarlowucf]

@zezha-msft Ah, I was only looking at the parameters. This should work then. Thank you.

[smarlowucf]

@zezha-msft I did a quick test with 32mb size and unfortunately it yields an error. </Message><MaxLimit>4194304</MaxLimit></Error>. So it seems like 4mb is a hard limit.

Edit: In the comments of the class it mentions The maximum page size the service supports is 4MB.

[zezha-msft]

Hi @smarlowucf, yes, there is indeed a limit for page blobs. The limit for block blobs is 100MB.

[smarlowucf]

Okay, thanks.