proxy.golang.org: proxy reporting unpublished version

[knusbaum]

What version of Go are you using (go version)?

$ go version
go version go1.16.4 darwin/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/kyle.nusbaum/Library/Caches/go-build"
GOENV="/Users/kyle.nusbaum/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/kyle.nusbaum/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/kyle.nusbaum/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/Users/kyle.nusbaum/sdk/go1.16.4"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/Users/kyle.nusbaum/sdk/go1.16.4/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.16.4"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/kyle.nusbaum/Documents/CodeBase/testgover/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/8x/bn6z9tk109n_hvbhl8fwdmpw0000gn/T/go-build643383583=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

When downloading the gopkg.in/DataDog/dd-trace-go.v1@latest, the proxy reports a version v1.31.0 which was never published, resulting in build failures.

Notice the v1.31.0 in this list:

$ go list -m --versions gopkg.in/DataDog/dd-trace-go.v1
gopkg.in/DataDog/dd-trace-go.v1 v1.0.0 v1.0.111-unstable v1.1.0 v1.1.1 v1.2.0 v1.2.1 v1.2.2 v1.2.3 v1.3.0 v1.4.0 v1.5.0 v1.6.0 v1.7.0 v1.8.0 v1.9.0 v1.10.0 v1.11.0 v1.12.0 v1.12.1 v1.13.0 v1.13.1 v1.14.0 v1.15.0 v1.16.0 v1.16.1-rc.1 v1.16.1 v1.17.0 v1.18.0 v1.19.0-rc.1 v1.19.0 v1.20.0-rc.1 v1.20.0 v1.20.1 v1.21.0-rc.1 v1.21.0-rc.2 v1.21.0-rc.3 v1.21.0 v1.21.1-rc.1 v1.22.0-rc.1 v1.22.0 v1.23.0-rc.1 v1.23.0 v1.23.1 v1.23.2 v1.23.3 v1.24.0-rc.1 v1.24.0-rc.2 v1.24.0-rc.3 v1.24.0-rc.4 v1.24.0 v1.24.1 v1.25.0-rc1 v1.25.0-rc2 v1.25.0-rc3 v1.25.0 v1.26.0-rc.1 v1.26.0-rc.2 v1.26.0 v1.27.0-rc.1 v1.27.0-rc.2 v1.27.0-rc.3 v1.27.0-rc.4 v1.27.0 v1.27.1-alpha.1 v1.27.1 v1.28.0-rc.1 v1.28.0-rc.2 v1.28.0 v1.29.0-alpha.1 v1.29.0-rc.1 v1.29.0 v1.29.1 v1.29.2 v1.29.3-rc.1 v1.30.0-rc.1 v1.30.0 v1.31.0 v1.32.0-alpha.1 v1.32.0-alpha.2 v1.32.0-alpha.3 v1.32.0-alpha.4 v1.32.0-alpha.5 v1.32.0-alpha.6 v1.32.0-alpha.7 v1.32.0-alpha.8 v1.32.0-alpha.9 v1.32.0-alpha.10 v1.32.0-alpha.11 v1.32.0-alpha.12 v1.32.0-alpha.13

With GOPROXY=direct, the correct latest version v1.30.0 is displayed:

$ GOPROXY=direct go list -m --versions gopkg.in/DataDog/dd-trace-go.v1
gopkg.in/DataDog/dd-trace-go.v1 v1.0.0 v1.1.0 v1.1.1 v1.2.0 v1.2.1 v1.2.2 v1.2.3 v1.3.0 v1.4.0 v1.5.0 v1.6.0 v1.7.0 v1.8.0 v1.9.0 v1.10.0 v1.11.0 v1.12.0 v1.12.1 v1.13.0 v1.13.1 v1.14.0 v1.15.0 v1.16.0 v1.16.1-rc.1 v1.16.1 v1.17.0 v1.18.0 v1.19.0-rc.1 v1.19.0 v1.20.0-rc.1 v1.20.0 v1.20.1 v1.21.0-rc.1 v1.21.0-rc.2 v1.21.0-rc.3 v1.21.0 v1.21.1-rc.1 v1.22.0-rc.1 v1.22.0 v1.23.0-rc.1 v1.23.0 v1.23.1 v1.23.2 v1.23.3 v1.24.0-rc.1 v1.24.0-rc.2 v1.24.0-rc.3 v1.24.0-rc.4 v1.24.0 v1.24.1 v1.25.0-rc1 v1.25.0-rc2 v1.25.0-rc3 v1.25.0 v1.26.0-rc.1 v1.26.0-rc.2 v1.26.0 v1.27.0-rc.1 v1.27.0-rc.2 v1.27.0-rc.3 v1.27.0-rc.4 v1.27.0 v1.27.1-alpha.1 v1.27.1 v1.28.0-rc.1 v1.28.0-rc.2 v1.28.0 v1.29.0-alpha.1 v1.29.0-rc.1 v1.29.0 v1.29.1 v1.29.2 v1.29.3-rc.1 v1.30.0-rc.1 v1.30.0 v1.32.0-alpha.1 v1.32.0-alpha.2 v1.32.0-alpha.3 v1.32.0-alpha.4 v1.32.0-alpha.5 v1.32.0-alpha.6 v1.32.0-alpha.7 v1.32.0-alpha.8 v1.32.0-alpha.9 v1.32.0-alpha.10 v1.32.0-alpha.11 v1.32.0-alpha.12 v1.32.0-alpha.13

What did you expect to see?

Versions for which no tag has ever existed should not appear on proxy.golang.org

What did you see instead?

A version of the library which was never tagged appeared in the proxy.

[findleyr]

CC @katiehockman @hyangah @heschi

[dmitshur]

@knusbaum Is it a possibility that a v1.31.0 version tag was pushed (perhaps accidentally) and later deleted? If so, that would explain how the module mirror could've picked it up, even though it's not available in direct mode now.

The index.golang.org index suggests the index time for gopkg.in/DataDog/dd-trace-go.v1@v1.31.0 was 2021-04-22 05:05:43.373928 +0000 UTC.

If that is the explanation for what happened, then module retraction may be helpful.

[knusbaum]

@dmitshur Thanks for checking.

We have a policy against pushing and then deleting tags (for obvious reasons), so this should not have happened, and the index time doesn't correspond to a likely time period we would have tagged something.

Is there any solution besides pushing a tag for v1.31.0? Module retraction looks good, but my understanding is that it works with go1.16+. What will happen to users of previous go releases?

[dmitshur]

I posted a comment at DataDog/dd-trace-go#929 (comment) which contains more relevant information and may be helpful, have you already seen it?

Another path for the dd-trace-go project may be to push a newer release like v1.31.1 or v1.32.0, which will become the new "latest" stable version.

Since we haven't been able to find evidence of a bug in proxy.golang.org behavior, I think we can close this issue, and issue DataDog/dd-trace-go#929 can be used for finding the optional resolution for that project. (The Go project doesn't use the issue tracker for questions, please see https://golang.org/wiki/Questions for a list of better places.)