net/http: HTTP client does not preserve headers when redirected

[fsufitch]

The http library in Go 1.7 HTTP client does not handle preserving headers correctly when it gets redirected. If a HTTP request gets redirected, the second request to the new location does not contain all the headers the previous request did.

This is not an issue in Go 1.8. Running the below steps to test it in 1.8 results in no problems.

What version of Go are you using (go version)?

go version go1.7.5 darwin/amd64

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/imuser/goenv"
GORACE=""
GOROOT="/Users/imuser/go1.7/go"
GOTOOLDIR="/Users/imuser/go1.7/go/pkg/tool/darwin_amd64"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/m2/907y66gx3v56c6njghgyvlx80000gq/T/go-build725648257=/tmp/go-build -gno-record-gcc-switches -fno-common"
CXX="clang++"
CGO_ENABLED="1"

What did you do?

Minimal repo with reproduction of the issue found here: https://github.com/fsufitch/go-redirbug

1. Start a server with go run server.go (in the server/ folder). This is a simple server that redirects from /foo to /foo/. It responds to /foo/ by checking the "Authorization" header for a valid value, and prints a result appropriate to that comparison.
2. Run the client with go run client.go (in the client/ folder). This is a simple client that queries the aforementioned server's /foo endpoint, with the correct authorization.

With unit tests:

Run the client's tests using go test in the client/ folder.

What did you expect to see?

The client should successfully complete the query to the server, getting redirected to /foo/ and providing the correct header, leading to an OK response from the server.

With unit tests:

When running the tests, both tests (TestQueryNoSlash and TestQueryWithSlash) pass.

What did you see instead?

The request that the /foo/ endpoint on the server receives does not include an "Authorization" header. This causes the server to reply with a 401 Unauthorized to the request instead of the expected OK.

With unit tests:

TestQueryNoSlash (which includes the redirect) fails, while TestQueryWithSlash (which does not redirect) does not.

[bradfitz]

So, use Go 1.8?

This doesn't qualify as something we'd backport.

[fsufitch]

I filed this issue as 1.7 is still advertised as a "stable" download (per https://golang.org/dl/), and could not find this behavior documented anywhere else (fix or not), which made me think "bug". If the team wishes not to fix it, that is your prerogative. In my particular use case, upgrading 1.7 -> 1.8 in the deployment pipeline is likely not something that will happen in the near term, so a workaround (or bugfix) is required.

Also, tracking down this problem was a massive pain, and even having a "Won't fix" bug report on it would have helped, so here I am. :)

[mvdan]

@fsufitch I believe the policy is to only backport fixes to the previous major stable (in this case 1.7) if they're security or otherwise critical issues. See https://golang.org/doc/devel/release.html.

Unless I'm missing something, that policy should be enough to tell you that this is a "won't fix" for what you're asking.

[fsufitch]

@mvdan Makes sense. I was not aware there was no support for previous versions aside from critical security stuff. Since this is not a critical security thing I can understand classifying it as "won't fix".

Thanks!

[bradfitz]

There have been dozens of bug reports about this in the past, going back to Go 1.0. See #4800 and all the dup bugs that reference it.

We documented this in the Go 1.8 release notes:

https://golang.org/doc/go1.8#net_http

The Client now copies most request headers on redirect. See the documentation on the Client type for details.

The answer is to use Go 1.8.