Azure Blob Storage: "TypeError: '<' not supported between instances of 'int' and 'NoneType'"

[AdmiralGT]

django-storages version: 1.12.3
Django version: 3.2.10
azure-storage-blob version: 12.9.0
azure-core version: 1.21.1

I'm serving a simple FileResponse and get the following error

2021-12-12T13:36:51.739891275Z Traceback (most recent call last):
2021-12-12T13:36:51.739908176Z   File "/opt/python/3.8.6/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 136, in handle
2021-12-12T13:36:51.739920476Z     self.handle_request(listener, req, client, addr)
2021-12-12T13:36:51.739931676Z   File "/opt/python/3.8.6/lib/python3.8/site-packages/gunicorn/workers/sync.py", line 182, in handle_request
2021-12-12T13:36:51.739943076Z     resp.write_file(respiter)
2021-12-12T13:36:51.739953676Z   File "/opt/python/3.8.6/lib/python3.8/site-packages/gunicorn/http/wsgi.py", line 385, in write_file
2021-12-12T13:36:51.739964877Z     if not self.sendfile(respiter):
2021-12-12T13:36:51.739975477Z   File "/opt/python/3.8.6/lib/python3.8/site-packages/gunicorn/http/wsgi.py", line 355, in sendfile
2021-12-12T13:36:51.739986577Z     if not util.has_fileno(respiter.filelike):
2021-12-12T13:36:51.739997177Z   File "/opt/python/3.8.6/lib/python3.8/site-packages/gunicorn/util.py", line 569, in has_fileno
2021-12-12T13:36:51.740008278Z     if not hasattr(obj, "fileno"):
2021-12-12T13:36:51.740019178Z   File "/tmp/8d9bd73cfc52ab6/antenv/lib/python3.8/site-packages/django/core/files/utils.py", line 38, in <lambda>
2021-12-12T13:36:51.740031578Z     fileno = property(lambda self: self.file.fileno)
2021-12-12T13:36:51.740042278Z   File "/tmp/8d9bd73cfc52ab6/antenv/lib/python3.8/site-packages/django/core/files/utils.py", line 38, in <lambda>
2021-12-12T13:36:51.740053978Z     fileno = property(lambda self: self.file.fileno)
2021-12-12T13:36:51.740064579Z   File "/tmp/8d9bd73cfc52ab6/antenv/lib/python3.8/site-packages/storages/backends/azure_storage.py", line 47, in _get_file
2021-12-12T13:36:51.740075879Z     download_stream.download_to_stream(file, max_concurrency=1)
2021-12-12T13:36:51.740086679Z   File "/tmp/8d9bd73cfc52ab6/antenv/lib/python3.8/site-packages/azure/storage/blob/_download.py", line 635, in download_to_stream
2021-12-12T13:36:51.740097979Z     self.readinto(stream)
2021-12-12T13:36:51.740108479Z   File "/tmp/8d9bd73cfc52ab6/antenv/lib/python3.8/site-packages/azure/storage/blob/_download.py", line 616, in readinto
2021-12-12T13:36:51.740119580Z     for chunk in downloader.get_chunk_offsets():
2021-12-12T13:36:51.740130480Z   File "/tmp/8d9bd73cfc52ab6/antenv/lib/python3.8/site-packages/azure/storage/blob/_download.py", line 123, in get_chunk_offsets
2021-12-12T13:36:51.740141780Z     while index < self.end_index:
2021-12-12T13:36:51.740152580Z TypeError: '<' not supported between instances of 'int' and 'NoneType'

I've reverted to django-storages 1.11 and that works fine.

Here's my AzureStorageClass

class AzureMediaStorage(AzureStorage):
    account_name = settings.AZURE_ACCOUNT_NAME
    account_key = settings.AZURE_STORAGE_KEY
    azure_container = settings.AZURE_MEDIA_CONTAINER
    expiration_secs = None
    overwrite_files = True

[dimbleby]

So far as I can see this code is just using the Azure API -

django-storages/storages/backends/azure_storage.py

Lines 44 to 46 in cf50a22

	download_stream = self._storage.client.download_blob(
	self._path, timeout=self._storage.timeout)
	download_stream.download_to_stream(file, max_concurrency=1)

.

Suggest reporting this at https://github.com/Azure/azure-sdk-for-python.

[dimbleby]

acknowledged upstream bug, this can be closed

[JeffreyCA]

The upstream issue was closed in August but the workaround presented doesn't seem to be working. I still get an IncompleteReadError error after modifying the django-storages Azure blob client with the request hook:

azure.core.exceptions.IncompleteReadError: ('Connection broken: IncompleteRead(8388608 bytes read, 25165824 more expected)', IncompleteRead(8388608 bytes read, 25165824 more expected))

After some investigation, I discovered that the reason why 1.11 doesn't have this issue is because it actually bypasses the CDN and streams the file directly from https://<storage-account>.blob.core.windows.net, which might not have been intended...

In 1.11, the call to download the file invokes .service, which bypasses the custom domain:

django-storages/storages/backends/azure_storage.py

Lines 169 to 178 in 3fbad31

	@property
	def service(self):
	if self._service is None:
	custom_domain = None
	if self.is_emulated:
	custom_domain = self.custom_domain
	self._service = self._blob_service(
	custom_domain=custom_domain,
	connection_string=self.connection_string)
	return self._service

django-storages/storages/backends/azure_storage.py

Lines 43 to 48 in 3fbad31

	self._storage.service.get_blob_to_stream(
	container_name=self._storage.azure_container,
	blob_name=self._path,
	stream=file,
	max_connections=1,
	timeout=self._storage.timeout)

In the latest version, that logic is not present anymore and fails when streaming from the Microsoft CDN.