Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: removing/switching tags on repository causes a go module checksum mismatch error #33969

Closed
rekby opened this issue Aug 30, 2019 · 7 comments
Closed

Comments

@rekby
Copy link

@rekby rekby commented Aug 30, 2019

What version of Go are you using (go version)?

$ go version
go version go1.12.9 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/rekby/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/rekby/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.12.9/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.12.9/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/rekby/tmp/gomod-test/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/8_/68c7r6rj6xg4fpjls__qtsjn18s1hz/T/go-build805875439=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

  1. Create repo github.com/rekby-forks/test-mod
  2. Create some file and tag it as semver, for example v0.0.2
  3. Create new go module
    go.mod
module rekby
go 1.12
require github.com/rekby-forks/test-mod v0.0.2

go.sum

package main

import tst "github.com/rekby-forks/test-mod"
import "fmt"

func main(){
    fmt.Println(tst.Test())
}
  1. go build. On it step created file go.sum
github.com/rekby-forks/test-mod v0.0.2 h1:ks22upaESPuYtz3t4FmkCKQCse5oMPs/gVTRavFVJlY=
github.com/rekby-forks/test-mod v0.0.2/go.mod h1:awjpQNGe4nVJmjCzjODnLPXGpp3acr9evJveSVjfBOY=
  1. go to repo github.com/rekby-forks/test-mod and remove tag, or remove and put for other commit.

  2. clean go cache

rm -rf /Users/rekby/Library/Caches/go-build/*; rm -rf ~/go/pkg/
  1. go build

What did you expect to see?

Build same binary, which as on step 4 (need commit exist in github.com/rekby-forks/test-mod)

What did you see instead?

if remove tag:

go: finding github.com/rekby-forks/test-mod v0.0.2
go: github.com/rekby-forks/test-mod@v0.0.2: unknown revision v0.0.2
go: error loading module requirements

if switch tag to other commit

go: finding github.com/rekby-forks/test-mod v0.0.2
go: downloading github.com/rekby-forks/test-mod v0.0.2
verifying github.com/rekby-forks/test-mod@v0.0.2: checksum mismatch
	downloaded: h1:PavIxURHqy6UUFJlEjFkimV/fJYUwPThVMa0MPvnBX0=
	go.sum:     h1:ks22upaESPuYtz3t4FmkCKQCse5oMPs/gVTRavFVJlY=

The commit exist in repo, it was tag change only. But I can't build binary and can't know what commit I need. Only way - manually (or self script) check every commit.

I think go.sum must contain info about revision of VSC (revision on svn, hash commit for git, hg, etc) and be independent from tags (which can beeasy change/remove).

@odeke-em odeke-em changed the title Bad external dependency description cmd/go: removing/switching tags on repository causes a go module checksum mismatch error Aug 31, 2019
@odeke-em

This comment has been minimized.

Copy link
Member

@odeke-em odeke-em commented Sep 1, 2019

Thank you for reporting this issue @rekby!

I am not sure what we can do here since the previous tag no longer exists upstream on the repository, meanwhile your go.mod/go.sum files contain the old checksums, thus you've declared that you want the non-existent commit to be built. This can happen in cases such as if someone
nefariously rewrites a commit and pushes up, so that error that you got is a safeguard.
Perhaps you should run go mod tidy on it to get back the old commit along with a go get -u?

However, I shall kindly loop in some go mod experts @bcmills @jayconrod.

@rekby

This comment has been minimized.

Copy link
Author

@rekby rekby commented Sep 2, 2019

When commit have no tag - go.sum contains part of commit hash - and it allow to computer find commit independent of tags.
Tags - more readable label for human and if go.sum contain tag - it is more readable, but lost accuracy - because the tag easy can rewrite by repo author and same tag point to other commit.

With commit it is more difficult - rewrite repo history and force push will change commit hashes in git and need full rebuild repo for svn.

I suggest add revision/commit hash to in additional to tag names, for example:

github.com/rekby-forks/test-mod v0.0.2(e9a220f8f3579aa9613bf755381b88f1fff60716) h1:ks22upaESPuYtz3t4FmkCKQCse5oMPs/gVTRavFVJlY=

it allow human read tag and human/computer read hash.

Then if tag point to different commit - go build can error with question what to do (change hash, change tag, stop) or fail with description with variants: change hash for use actual tagged version, change tag - for use previous commit.
Or it can show warning and use hash for checkout.

In any varian commit hash allow manual undestand what was previous state.
And prevent situation, when commit exist in repo, but no way to find it.

@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Sep 3, 2019

The go.mod and go.sum files are both written in terms of semantic versions, not commit hashes. That makes them — and the module proxy protocol — agnostic to the particular version-control system used to manage the code. (The commit hashes still leak through as pseudo-versions, but even then they're still represented as semantically-meaningful semver strings.)

The overall design of modules is that versions are immutable. A given version of a given module must always refer to the exact same contents. (The checksum database, for which support was added in Go 1.13, helps to ensure that property.) For that reason, we explicitly do not support changing semantic-version tags to point to different contents over time — if you need to tag a new release, it should have a new version number.

If you are worried about particular versions of your module dependencies disappearing or being deleted, one mitigation is to use a durable module mirror instead of fetching directly from upstream version control.

@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Sep 3, 2019

@rekby, did you encounter this problem in the wild, or is this concern mostly theoretical?

We've tried to communicate clearly that module versions must be immutable, but perhaps there are some loose ends in the documentation that we can clear up if we can figure out the source of confusion.

@rekby

This comment has been minimized.

Copy link
Author

@rekby rekby commented Sep 4, 2019

@bcmills it is mote theoretical problem - about repo owner.

For example I can change some tags in my small popular (near private) repo some times for ьн сщтмутшутсу if I think about only I use it. Or if I typo in tag (for example set tag v0.2 instead of v0.0.2). If I mistake and somebody add my repo from error tag v0.2 in short time, what tag existed - it can be problem for code consumer.

I did not encounded the problem in real wild.

@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Sep 4, 2019

You may safely change the tags on your repository as long as you are certain that nobody — including sum.golang.org — has fetched those tags yet.

If you accidentally typo a tag, such as v0.2.1 instead of v0.1.2, then the most robust way to fix it is to select and publish a higher version still.

@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Sep 4, 2019

I don't think there's anything more to be done about this. The entire purpose of the go.sum file and checksum database is to enforce that the mapping of versions to source code does not change over time.

@bcmills bcmills closed this Sep 4, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.