Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: cannot go get when all versions are retracted #42648

Open
hyangah opened this issue Nov 17, 2020 · 2 comments
Open

cmd/go: cannot go get when all versions are retracted #42648

hyangah opened this issue Nov 17, 2020 · 2 comments

Comments

@hyangah
Copy link
Contributor

@hyangah hyangah commented Nov 17, 2020

What version of Go are you using (go version)?

$ go version
go version devel +f9108d168e Mon Nov 16 17:46:12 2020 -0500 darwin/amd64

Does this issue reproduce with the latest release?

Reproducible only with tip that understands 'retract' rules.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/hakim/Library/Caches/go-build"
GOENV="/Users/hakim/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/hakim/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/hakim/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/Users/hakim/go_tip/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/Users/hakim/go_tip/go/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="devel +f9108d168e Mon Nov 16 17:46:12 2020 -0500"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/bw/6r6k9d113sv1_vvzk_1kfxbm001py5/T/go-build001158049=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Module name: github.com/hyangah/retracttest/a
Retracted all released versions [v1.0.0,v1.0.1] - as noted in go.mod@v1.0.1

What did you expect to see?

Retract only the released versions of a module. I thought I could let future users pick up the latest pseudo-versions after retraction - #41700.

What did you see instead?

I ended up making the entire module unavailable.

$ go get -x github.com/hyangah/retracttest/a@latest
# get https://proxy.golang.org/github.com/@v/list
# get https://proxy.golang.org/github.com/hyangah/retracttest/a/@v/list
# get https://proxy.golang.org/github.com/hyangah/@v/list
# get https://proxy.golang.org/github.com/hyangah/retracttest/@v/list
# get https://proxy.golang.org/github.com/@v/list: 410 Gone (0.283s)
# get https://proxy.golang.org/github.com/hyangah/retracttest/@v/list: 200 OK (0.288s)
# get https://proxy.golang.org/github.com/hyangah/retracttest/@latest
# get https://proxy.golang.org/github.com/hyangah/@v/list: 410 Gone (0.312s)
# get https://proxy.golang.org/github.com/hyangah/retracttest/a/@v/list: 200 OK (0.320s)
# get https://proxy.golang.org/github.com/hyangah/retracttest/a/@latest
# get https://proxy.golang.org/github.com/hyangah/retracttest/@latest: 200 OK (0.044s)
# get https://proxy.golang.org/github.com/hyangah/retracttest/a/@latest: 200 OK (0.144s)
go get: module github.com/hyangah/retracttest@latest found (v0.0.0-20201117000603-58d79ad1d7ed), but does not contain package github.com/hyangah/retracttest/a

go get with GOPROXY=direct works though, so it could be bad interaction with proxy.golang.org.

And go list result is confusing:

With default GOPROXY:

$ go list -m --versions -json github.com/hyangah/retracttest/a
{
	"Path": "github.com/hyangah/retracttest/a",
	"Version": "v1.0.1",
	"Time": "2020-11-16T22:37:32Z",
	"Dir": "/Users/hakim/go/pkg/mod/github.com/hyangah/retracttest/a@v1.0.1",
	"GoMod": "/Users/hakim/go/pkg/mod/cache/download/github.com/hyangah/retracttest/a/@v/v1.0.1.mod",
	"GoVersion": "1.15"
}

$ go list -m --versions -json github.com/hyangah/retracttest/a@latest
go list -m: module github.com/hyangah/retracttest/a: no matching versions for query "latest"

With GOPROXY=direct:

$ GOPROXY=direct go list -m --versions -json github.com/hyangah/retracttest/a
{
	"Path": "github.com/hyangah/retracttest/a",
	"Version": "v1.0.1",
	"Time": "2020-11-16T22:37:32Z",
	"Dir": "/Users/hakim/go/pkg/mod/github.com/hyangah/retracttest/a@v1.0.1",
	"GoMod": "/Users/hakim/go/pkg/mod/cache/download/github.com/hyangah/retracttest/a/@v/v1.0.1.mod",
	"GoVersion": "1.15"
}

$ GOPROXY=direct tipgo list -m --versions -json github.com/hyangah/retracttest/a@latest
{
	"Path": "github.com/hyangah/retracttest/a",
	"Version": "v0.0.0-20201117000603-58d79ad1d7ed",
	"Time": "2020-11-17T00:06:03Z",
	"Dir": "/Users/hakim/go/pkg/mod/github.com/hyangah/retracttest/a@v0.0.0-20201117000603-58d79ad1d7ed",
	"GoMod": "/Users/hakim/go/pkg/mod/cache/download/github.com/hyangah/retracttest/a/@v/v0.0.0-20201117000603-58d79ad1d7ed.mod",
	"GoVersion": "1.15"
}

cc @jayconrod @bcmills @matloob @heschik @katiehockman

@bcmills bcmills added this to the Go1.16 milestone Nov 17, 2020
@hyangah
Copy link
Contributor Author

@hyangah hyangah commented Nov 17, 2020

This is because proxy.golang.org isn't yet using go 1.16 that understands version retraction. According to #24031 (comment),

If no pre-release version is available, the go command either requests a version from the proxy's $module/@latest endpoint or derives a pseudo-version from the module repository's default branch (in direct mode).

And according to golang.org/ref/mod

$base/$module/@latest
Returns JSON-formatted metadata about the latest known version of a module in the same format as $base/$module/@v/$version.info. The latest version should be the version of the module that the go command should use if $base/$module/@v/list is empty or no listed version is suitable. This endpoint is optional, and module proxies are not required to implement it.

As go1.16 rolls out, proxy maintainers need to make sure either to implement the end point correctly, or let the go command fallback to direct (if users chose to do so) by responding with 404/410.

It would be nice if the go command's error message gets improved though.

@jayconrod jayconrod self-assigned this Nov 17, 2020
@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Nov 17, 2020

So it looks like the proxy returns the highest release version from the @latest point, rather than a pseudo-version. Since v1.0.1 is retracted, the go command has no suitable version to use.

$ curl -L https://proxy.golang.org/github.com/hyangah/retracttest/a/@latest
{"Version":"v1.0.1","Time":"2020-11-16T22:37:32Z"}

The first error message is a bit obscure, but I think it's reasonable. The go command failed to find a suitable versions of the submodule, but it did find a version of the parent module, and that version didn't contain the package. You might see this if the author split the repo up into submodules, decided it was a bad idea, and retracted all versions of the submodules.

module github.com/hyangah/retracttest@latest found (v0.0.0-20201117000603-58d79ad1d7ed), but does not contain package github.com/hyangah/retracttest/a

The second error message is less helpful. This one should explicitly mention that all versions were retracted.

go list -m: module github.com/hyangah/retracttest/a: no matching versions for query "latest"

I think a couple changes to the proxy are needed though:

  • When fetching the version list, the proxy should use go list -m -versions -retracted $mod. The new -retracted flag will include retracted versions in the list.
  • For the @latest endpoint, the proxy should always return a pseudo-version. That can be used when all versions from the @v/list endpoint are retracted or excluded.
    • One way to get this version with GOPROXY=direct would be to run go list -m -versions -retracted $mod as above, then create a temporary go.mod file with an exclude directive for each version, then run go list -m $mod@latest. #42545 would provide an easier way to get that, but we won't be able to implement that for 1.16.

@hyangah Does this sound reasonable? Should I open a separate issue for the proxy changes?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.