HttpClientHandler.AutomaticDecompression only decompresses one layer

[erikness]

Issue description

In HttpClientHandler, you are allowed to set AutomaticDecompression (values are GZip, Deflate, Brotli, None, or a combination of those). The idea is that .NET will automatically decompress HTTP responses for you.

Compression on the HTTP response body is defined by by the "Content-Encoding" header. However, this can have multiple values if the content was compressed multiple times:

Content-Encoding: gzip, br

They are listed in the order that they were applied. That is, this server applied GZip compression to some content, and then applied Brotli compression to that gzipped content.

Both the .NET Framework and .NET Core source code have a peculiar behavior though: if there are multiple specified, it only decompresses the last - or outermost - compression. So in this case, it would decompress the Brotli-encoded content, and then pass along the GZipped content to the user. It would also strip the word "brotli" from the Content-Encoding header, so the user would se it as Content-Encoding: gzip.

Link to .NET Core source code that does this

Link to .NET Framework source code that does this

(I also made a small repro, ask if you need it)

As far as I know this isn't incorrect - .NET is allowed to limit itself to 1 decompression, I guess - but it's not documented anywhere. The right place might be in the docs for HttpClientHandler, or HttpClientHandler.AutomaticDecompression. Though it would also be nice if the .NET team provided a reason why .NET chooses to do this.

Target framework

All versions of Core/Framework/Standard, as far as I know.

[mairaw]

Thanks for your feedback @erikness. @karelz @davidsh can you take a look at this feedback?

[davidsh]

Thanks for your feedback @erikness. @karelz @davidsh can you take a look at this feedback?

This is good feedback. Both .NET Framework and .NET Core have historically always worked this way. We haven't seen any other customer reports about using multiple compression schemes for HTTP content. I don't think we ever considered scenarios with multiple values in the 'Content-Encoding' header.

At a minimum, yes, we can document the current behavior. But it might be useful for you to open an issue about this here: https://github.com/dotnet/runtime/issues

Perhaps, a future version of .NET (.NET Core), can support multiple compression layers.

cc: @dotnet/ncl

[erikness]

Thanks, David! I'll open up an issue there too, however, it's unclear to me how important multiple compression layers are.

Interesting thing to note: before .NET Core supported Brotli, it only had GZip and Deflate, which are 2 different formats for the same underlying algorithm (so why would anyone apply it twice?).

[erikness]

One more thing that should be documented: when the user sets AutomaticDecompression to some values, .NET will send an Accept-Encoding header for each of those values in every request sent from that handler. This was actually a point of confusion for another team I worked with recently.

.NET Framework source link
.NET Core source link