fix resumable downloads crash when gzip

[memsharded]

Changelog: Bugfix: Resumable download introduced a bug when there is a fronted (like Apache) to Artifactory or other server that gzips the returned files, returning an incorrect Content-Length header that doesn't match the real content length.
Docs: Omit

Fix: #6984

This is a fix to #6984, but it would be good to implement something more robust that handles the compression from servers in a better way.

[flashdagger]

Should we also consider other possible compression algorithms?

Suggested change

	gzip = (response.headers.get("content-encoding") == "gzip")
	is_compressed = (response.headers.get("content-encoding") in {"gzip", "deflate", "br"})

You could test them e.g. here: https://httpbin.org/#/Response_formats

As an alternative you could compute the total_length with len(response.content) in those cases, ignoring the Content-Length from headers...

[memsharded]

I'd prefer to just "revert" that regression, and we could open a new ticket for improving this for next releases. These other compressions types haven't been an issue so far (they would have raised exceptions in the past, just by using them), so I think this is ok as-is.

[jgsogo]

It looks like we are not the only ones with this problem and it is something we cannot fix from the client-side [1] (unless we do not request gzip from the client?). We can merge this one if we are going to release 1.25 to fix the bug/regression (excluding other compression types as @flashdagger suggests probably is needed).

---

[1] https://serverfault.com/questions/183843/content-length-not-sent-when-gzip-compression-enabled-in-apache

[flashdagger]

So my understanding is this:

You can only check if you received the full data requested if:

• Content-Encoding is "identity" (the response default)
• Content-Length is sent in response

If the content body is sent compressed (e.g. using gzip) Content-Length represents the compressed size (what was sent over the wire) or is even omitted. In that case transmission should fail anyway if data is missing. Not a case we should consider. So we cannot resume compressed responses since we don't read from the raw socket and let the requests library do the decompression.