net/http: ReadResponse rejects duplicate
Transfer-Encoding: chunked headers
Since Go 1.15 (commit d5734d4), http.ReadResponse (and therefore http.Transport.RoundTrip) returns an error if the response has more than one Transfer-Encoding header.
This makes it impossible to download content from certain sites. For example, attempting to download from https://sis.cat.com/ returns the error
If I understand the discussion around the change correctly, there are security issues related to multiple Transfer-Encoding headers on a request. But this is on a response.
Could we be a little less strict for responses? Or allow multiple Transfer-Encoding headers if they are all "chunked"?
The text was updated successfully, but these errors were encountered:
I did some further investigation.
My testing included sending back multiple chunked headers on a single line as well as with multiple headers. I also sent back
Chrome didn't care what came back and always managed to print my response. They are just playing fast and loose to make the user experience appear to work in the face of all types of possible errors.
Curl didn't care how many chunked encodings were sent, but did fail when
Coalescing multiple chunked encodings won't hurt. The downside is that these broken configured servers are allowed to go on their merry way.
That is vexing.
If curl and Chrome need to "make the user experience appear to work", and if the user can't between something that works vs "appears to work", should we as users of the Go library take that to mean that the world's waiting on the Go http library to enforce the spec?
Or could there be some other way to compile the Go library to "appear to work" as well as Chrome / curl?
As it currently is, I've downgraded to 1.14.14 and that doesn't seem like the best of alternatives either.