cmd/go: work around insecure redirects for domains hosted on GitHub Pages

[rogpeppe]

What version of Go are you using (go version)?

$ go version
go version devel +2e4edf4697 Sun May 12 07:14:09 2019 +0000 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/rog/.cache/go-build"
GOENV="/home/rog/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/rog/src/go"
GOPROXY="direct"
GOROOT="/home/rog/go"
GOSUMDB="off"
GOTMPDIR=""
GOTOOLDIR="/home/rog/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/rog/src/work/quicktest/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build245885201=/tmp/go-build -gno-record-gcc-switches"

What did you do?

 cd $(mktemp -d)
 go mod init m
 go get googlemaps.github.io/maps

What did you expect to see?

I'd expect the command to succeed. It works on Go 1.12.

What did you see instead?

$ go get googlemaps.github.io/maps
go get googlemaps.github.io/maps: unrecognized import path "googlemaps.github.io/maps" (https fetch: Get http://googlemaps.github.io/maps/?go-get=1: redirected from secure URL https://googlemaps.github.io/maps?go-get=1 to insecure URL http://googlemaps.github.io/maps/?go-get=1)

[FiloSottile]

This is working as intended following #29618. https://googlemaps.github.io/maps?go-get=1 is serving a redirect to an HTTP link, which is insecure, so requires the -insecure flag.

/cc @bcmills who is tracking impact of that change.

[FiloSottile]

(It would be nice to ask GitHub support why on Earth they are doing that, btw. It's bad for any client, not just the go command.)

[domesticmouse]

I'm curious what I'm doing wrong here. (I'm the Googler responsible for the Go googlemaps project.)

I set up redirects in https://github.com/googlemaps/googlemaps.github.io/blob/master/maps/index.html to point to https URLs.

Hints on how I can fix this gladly accepted.

[bcmills]

There is an interesting clue in isaacs/github#289 (comment):

GitHub Support wrote:

This redirection quirk that you've discovered is a known issue, and relates to trailing slashes and how our redirects work at the moment. Making a request to a URL without a trailing slash will cause redirections to run in this order:

request to original URL
301 to HTTPS version of original URL
301 to HTTP version with added trailing slash
301 to HTTPS version with added trailing slash
Always including the trailing slash will allow you to skip the extra redirects:

request to URL with trailing slash
301 to HTTPS version of original URL
We have an issue open internally to hopefully improve this situation, but investigations have demonstrates that it is a non-trivial issue to resolve.

[bcmills]

Reopening to consider workarounds, but I'd really rather GitHub fix their redirects instead.

[rsc]

If -insecure is not being used and we see https://foo redirect to http://foo/ (that is, the only deltas are s|https|http|; s|$|/|) can we quietly put the https back instead of failing?

[bcmills]

Yep, that seems like a plausible workaround — it's small, doesn't hard-code information specific to GitHub, and doesn't require that we change the initial URL.