Don't send Content-Encoding header with streamed responses

[dkliban]

aiohttp automatically enflates gzipped responses. Pulp clients always receive
uncompressed responses when requesting on_demand content.

fixes: #9213

[pulpbot]

Attached issue: https://pulp.plan.io/issues/9213

[bmbouter]

This makes sense for gzip, but what about other values of Content-Encoding besides gzip?

[dralley]

@bmbouter You're referring to the changelog entry?

[bmbouter]

@bmbouter You're referring to the changelog entry?

I mean functionally. We know aiohttp decompresses gzip, but what about others? If aiohttp doesn't decompress it then removing the content encoding header would be incorrect. (I have no idea what aiohttp does).

[dkliban]

@bmbouter You're referring to the changelog entry?

I mean functionally. We know aiohttp decompresses gzip, but what about others? If aiohttp doesn't decompress it then removing the content encoding header would be incorrect. (I have no idea what aiohttp does).

All of the 4 possible values for Content-Encoding imply compression[0]. aiohttp decompresses 'gzip' and 'deflate' by default. 'brotli' encoding support can be enabled by installing 'brotlipy'[1]. 'encode' does not seem to be used much by servers due to a pattent issue. It doesn't look like aiohttp has support for auto inflating the response when the Content-Encoding is 'compress'.

[0] https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Encoding#directives
[1] https://docs.aiohttp.org/en/stable/client_quickstart.html?highlight=deflate#binary-response-content

[bmbouter]

To be sure we're not regressing, what if we only discard the Content-Encoding header if it's value is gzip?

[dralley]

@bmbouter Because aiohttp can't handle other compression types, I'm pretty sure it would set Accept-Encoding such that the server would never provide them. Although we should check that we're not forwarding those headers from the client.

[bmbouter]

@bmbouter Because aiohttp can't handle other compression types, I'm pretty sure it would set Accept-Encoding such that the server would never provide them. Although we should check that we're not forwarding those headers from the client.

Does it also accept deflate and optionally brotli too? https://docs.aiohttp.org/en/stable/client_quickstart.html?highlight=deflate#binary-response-content

[dralley]

https://github.com/aio-libs/aiohttp/blob/09ac1cbb4c07b4b3e1872374808512c9bc52f2ab/aiohttp/client_reqrep.py#L85-L86

[bmbouter]

@dralley so what would you recommend given that?

[dralley]

@bmbouter If aiohttp is setting the Accept-Encoding header on the request, then it shouldn't receive any format that aiohttp cannot deal with - so as long as we are not forwarding Accept-Encoding from the client's original request, stripping the Content-Encoding header would be fine for all possible encoding types.

[bmbouter]

@bmbouter If aiohttp is setting the Accept-Encoding header on the request, then it shouldn't receive any format that aiohttp cannot deal with - so as long as we are not forwarding Accept-Encoding from the client's original request, stripping the Content-Encoding header would be fine for all possible encoding types.

Well said. I agree completely.

@dkliban can you confirm via local testing that your patch is resolving the issue? You were able to reproduce the issue right?

[bmbouter]

Maybe this?

Suggested change

	Fixed bug where uncompressed on_demand content was streamed with "Content-Encoding: gzip" header.
	Fixed bug where ``pulpcore-content`` decompressed data while incorrectly advertising to clients it was still compressed via the ``Content-Encoding: gzip`` header.

[dkliban]

@bmbouter If aiohttp is setting the Accept-Encoding header on the request, then it shouldn't receive any format that aiohttp cannot deal with - so as long as we are not forwarding Accept-Encoding from the client's original request, stripping the Content-Encoding header would be fine for all possible encoding types.

Well said. I agree completely.

@dkliban can you confirm via local testing that your patch is resolving the issue? You were able to reproduce the issue right?

I reproduced the issue locally and confirmed that the patch resolves it.

[bmbouter]

Thanks!