Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: option to allow Trailer header without chunked transfer encoding in HTTP response #27197

Closed
VictoriaRaymond opened this issue Aug 24, 2018 · 9 comments

Comments

Projects
None yet
6 participants
@VictoriaRaymond
Copy link

commented Aug 24, 2018

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

1.10

Does this issue reproduce with the latest release?

Yes, as the code is not changed.

What operating system and processor architecture are you using (go env)?

Mac OS 10.13 (don't matter actually)

What did you do?

Read HTTP response via http.ReadResponse(). The HTTP response contains a "Trailer" header but no "Transfer-Encoding" head. This leads to an error during header parsing.

It is not possible to reproduce using the original URL for http request/response, as the URL requires authentication in advance. Here is a simplified code using static real response.

https://play.golang.org/p/nuL0CnYF1tu

What did you expect to see?

Since response body is not chunked in such case, the "Trailer" header can be considered (as least optionally) as a "no-op". It would be nice that net/http can ignore such error (by default, or with an option), and let caller to decide whether the response is valid or not.

RFC 7230 doesn't say anything regarding such case. It only mentions positive case when Transfer-Encoding is used.

Reference:
https://tools.ietf.org/html/rfc7230#section-4.4
https://tools.ietf.org/html/rfc7230#section-4.1.2

What did you see instead?

ReadResponse() returns ErrUnexpectedTrailer, which is originated from here: https://github.com/golang/go/blob/master/src/net/http/transfer.go#L761

For the moment, the caller has no choice but to discard the HTTP response.

@meirf

This comment has been minimized.

Copy link
Contributor

commented Aug 25, 2018

@VictoriaRaymond, if we removed this error, at the very least we would want to know some more detail in terms of prioritizing the change. Can you give more details on why the server is setting trailer without chunked encoding?

@meirf meirf added this to the Unplanned milestone Aug 25, 2018

@Dreamacro

This comment has been minimized.

Copy link

commented Aug 25, 2018

@meirf This error occurs when downloading application on App Store with macOS. Although this request may be non-standard, there is no way to get response (return nil) when it's error. So i hope golang can provide a solution that can still access Response when ReadResponse get error

@meirf

This comment has been minimized.

Copy link
Contributor

commented Aug 25, 2018

@Dreamacro, thanks. do you have a url we can look at?

@Dreamacro

This comment has been minimized.

Copy link

commented Aug 25, 2018

@meirf

The error was found in the App Store, such as download https://itunes.apple.com/cn/app/autodesk-sketchbook/id863486266?l=en&mt=12
Host is osxapps.itunes.apple.com, which looks like an API for verifying signatures
The returned headers can be found at https://play.golang.org/p/nuL0CnYF1tu

@meirf

This comment has been minimized.

Copy link
Contributor

commented Aug 25, 2018

If you have a momemt, can you include an executable example of the code you're running against the url? (I've tried a few different ways to get the response, but I'm not seeing the exact headers you are seeing. Perhaps the response is different due to geography; I'm in the US, your profile shows China. There could be totally different server infrastructure. For example, my requests are being served by an akamai cdn.)

@bradfitz

This comment has been minimized.

Copy link
Member

commented Aug 26, 2018

Thanks for the https://play.golang.org/p/nuL0CnYF1tu demo.

I don't want to add more configuration option complexity. And I agree it's a little lame that users can't read these responses. This is easy enough to fix once it's decided what to do. I suppose we can just keep the Trailer value in the Response.Header map but not populate Response.Trailer.

@bradfitz bradfitz modified the milestones: Unplanned, Go1.12 Aug 26, 2018

@bradfitz bradfitz added the NeedsFix label Aug 26, 2018

@BigMikes

This comment has been minimized.

Copy link
Contributor

commented Oct 28, 2018

I would love to fix this. Can I try?

@bradfitz

This comment has been minimized.

Copy link
Member

commented Oct 28, 2018

Sure

@gopherbot

This comment has been minimized.

Copy link

commented Oct 29, 2018

Change https://golang.org/cl/145398 mentions this issue: net/http: not to return error with Trailer header when no chunked

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.