Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sum.golang.org: go-get gitern.com/xxx/repo.git 410 gone #40358

Closed
arnottcr opened this issue Jul 23, 2020 · 15 comments
Closed

sum.golang.org: go-get gitern.com/xxx/repo.git 410 gone #40358

arnottcr opened this issue Jul 23, 2020 · 15 comments

Comments

@arnottcr
Copy link
Contributor

@arnottcr arnottcr commented Jul 23, 2020

What version of Go are you using (go version)?

$ go version
go version go1.14.6 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/user/.cache/go-build"
GOENV="/home/user/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/user/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/tmp/trash/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build897724545=/tmp/go-build -gno-record-gcc-switches"

What did you do?

go get gitern.com/urandom/errors.git

What did you expect to see?

go: downloading gitern.com/urandom/errors.git v0.0.0-20200723020514-49681dcbcc21

What did you see instead?

go: downloading gitern.com/urandom/errors.git v0.0.0-20200723020514-49681dcbcc21
go get gitern.com/urandom/errors.git@master: gitern.com/urandom/errors.git@v0.0.0-20200723020514-49681dcbcc21: verifying module: gitern.com/urandom/errors.git@v0.0.0-20200723020514-49681dcbcc21: reading https://sum.golang.org/lookup/gitern.com/urandom/errors.git@v0.0.0-20200723020514-49681dcbcc21: 410 Gone
        server response:
        not found: gitern.com/urandom/errors.git@v0.0.0-20200723020514-49681dcbcc21: invalid version: git fetch -f origin refs/heads/*:refs/heads/* refs/tags/*:refs/tags/* in /tmp/gopath/pkg/mod/cache/vcs/c58e8953608705577f340d5d716eb03f031f4f09a3698dde27ce667a4baf661a: exit status 128:
                fatal: unable to connect to gitern.com:
                gitern.com[0: 100.20.209.4]: errno=Connection refused

The repo appears correctly populated in ~/go/pkg/mod/cache, implying that proxy.golang.org is working, plus hitting vcs directly, via GOPRIVATE=gitern.com, works as expected.

@heschik
Copy link
Contributor

@heschik heschik commented Jul 23, 2020

You have GOPROXY="https://proxy.golang.org,direct", which means the go command will fall back to direct fetching if proxy.golang.org fails.

This is a strange one. It seems to be failing very consistently. Offhand the only thing I can think is that gitern.com is blocking the Google egress IPs that proxy.golang.org connects from. You may want to ask them if that rings any bells. I'll try to dig a little more when I get a chance.

@heschik
Copy link
Contributor

@heschik heschik commented Jul 23, 2020

@arnottcr
Copy link
Contributor Author

@arnottcr arnottcr commented Jul 23, 2020

cc @huumn

@huumn
Copy link

@huumn huumn commented Jul 23, 2020

I’m not blocking any ips explicitly. If they’re being blocked on my end they’re being blocked by AWS.

@huumn
Copy link

@huumn huumn commented Jul 23, 2020

All repos are private on gitern which could require additional config for go get?

Edit: Ah gitern only supports git over ssh to the repos. I suspect go get uses http.

@arnottcr
Copy link
Contributor Author

@arnottcr arnottcr commented Jul 23, 2020

Would go-get be falling back to direct for the zipball, which I have manually configured via config to use auth, then failing to find an entry in the sumdb?

@huumn
Copy link

@huumn huumn commented Jul 23, 2020

@arnottcr gitern does not support git over http currently (only ssh). I think go get defaults to http and might need to be told to use ssh.

@arnottcr
Copy link
Contributor Author

@arnottcr arnottcr commented Jul 23, 2020

The .git suffix causes the toolchain to try http and ssh (maybe git:// too), so i can confirm that part is working.

It would be nice to get meta header support for gitern, but that is out of scope here, and I still think there might be a go-get bug.

@hyangah
Copy link
Contributor

@hyangah hyangah commented Jul 23, 2020

For private repositories, please see this instruction.
https://golang.org/cmd/go/#hdr-Module_configuration_for_non_public_modules

@hyangah
Copy link
Contributor

@hyangah hyangah commented Jul 23, 2020

@arnottcr I just saw you already tried GOPRIVATE and it works. There is no way sum.golang.org can access private repositories (it shouldn't, right? it's private!) GOPRIVATE is the way to opting out of using proxy.golang.org and also sum.golang.org for the specified modules.

I will close this since this is working as intended.

@hyangah hyangah closed this Jul 23, 2020
@arnottcr
Copy link
Contributor Author

@arnottcr arnottcr commented Jul 23, 2020

This ux is really confusing. Can we have the toolchain detect these failurse and suggest GOPRIVATE? We use it at work, and I am familiar with what it does, but it was not clear why go get was failing.

@heschik
Copy link
Contributor

@heschik heschik commented Jul 23, 2020

No, especially since I still don't know why it's failing. Connection refused isn't a reasonable error to get for a private repository; it can't possibly know whether we can authenticate without accepting the connection. My best guess is still a misconfiguration on the origin side.

At the same time, there is no point spending time diagnosing it if it's just going to fail later, so I'm not going to investigate any further.

@arnottcr
Copy link
Contributor Author

@arnottcr arnottcr commented Jul 23, 2020

I guess my ask is when the module proxy falls back to direct, can we disable sumdb checking? I get that verification is important, but the ux is confusing, and if proxy.golang.org (or whatever module proxy you have configured) cannot reach out, why would we expect the sum.golang.org to do any better.

Since gitern may be making things more confusing, i am happy to repro with a private github repo.

@heschik
Copy link
Contributor

@heschik heschik commented Jul 23, 2020

Automatically disabling sumdb lookup would destroy the security model; any attacker that controlled the network could disable sumdb lookup by blocking access to proxy.golang.org.

Again, this is not a general problem with private modules. For example, I believe GitHub returns a warning about authentication that helps avoid confusion. The problem here is that network connectivity between proxy.golang.org and the origin is broken. There may be some room to improve the error message when sumdb fetching fails after a successful proxy download, but that would need to take into account the user's particular proxy/sumdb configuration and should definitely be discussed in a separate issue.

@arnottcr
Copy link
Contributor Author

@arnottcr arnottcr commented Jul 24, 2020

Automatically disabling sumdb lookup would destroy the security model; any attacker that controlled the network could disable sumdb lookup by blocking access to proxy.golang.org.

If you are concerned about this why even have the ,direct option? IIUC, it will always fail, since packages that do not exist in proxy.golang.org will not have entries in sum.golang.org. If we include ,direct it needs to do something, can you walk me through a ux that does?

Furthermore, if you are concerned with network level interception, you likely have larger problems.

Again, this is not a general problem with private modules. For example, I believe GitHub returns a warning about authentication that helps avoid confusion. The problem here is that network connectivity between proxy.golang.org and the origin is broken. There may be some room to improve the error message when sumdb fetching fails after a successful proxy download, but that would need to take into account the user's particular proxy/sumdb configuration and should definitely be discussed in a separate issue.

The ux for github private repos is not better, but this may be the anti-discovery mechanism at work:

[user@localhost ~]$ go get github.com/arnottcr/private
# cd .; git clone -- https://github.com/arnottcr/private /home/user/go/src/github.com/arnottcr/private
Cloning into '/home/user/go/src/github.com/arnottcr/private'...
fatal: could not read Username for 'https://github.com': terminal prompts disabled
package github.com/arnottcr/private: exit status 128

Also the toolchain disallows the use of github.com/owner/repo.git, so it does not directly translate: (really unsure why this is disallowed)

[user@localhost ~]$ go get github.com/arnottcr/private.git
package github.com/arnottcr/private.git: invalid version control suffix in github.com/ path
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.