Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: checksum mismatch with private dependent module with GIT LFS files #38941

Closed
pbrit opened this issue May 8, 2020 · 5 comments
Closed

cmd/go: checksum mismatch with private dependent module with GIT LFS files #38941

pbrit opened this issue May 8, 2020 · 5 comments

Comments

@pbrit
Copy link

@pbrit pbrit commented May 8, 2020

Summary

Due to this issue, the generation of the checksums for private modules depends on the availability of git-lfs binary.

I propose to pass GIT_LFS_SKIP_SMUDGE=1 to git binary when dependent modules are being fetched.

Important conditions for the repro:

  1. org/private-repo is a private repository
  2. org/private-repo has GIT LFS files
  3. git-lfs binary is available
  4. git-lfs filter is configured (see below).
$ git config -l 
filter.lfs.process=git-lfs filter-process
filter.lfs.required=true
filter.lfs.clean=git-lfs clean -- %f
filter.lfs.smudge=git-lfs smudge -- %f

What version of Go are you using (go version)?

$ go version
go version go1.14.2 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE="auto"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/vscode/.cache/go-build"
GOENV="/home/vscode/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GONOPROXY="github.com/org/private-repo"
GONOSUMDB="github.com/org/private-repo"
GOOS="linux"
GOPATH="/go"
GOPRIVATE="github.com/org/private-repo"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/workspaces/repo/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build765684387=/tmp/go-build -gno-record-gcc-switches"

What did you do?

$ go run main.go

What did you expect to see?

The binary being run.

What did you see instead?

verifying github.com/org/private-repo@v0.1.1: checksum mismatch
        downloaded: h1:VHvAPce3Ul6tvbVTQiXQtX6mQDntkwvStfgCAzQrrAY=
        go.sum:     h1:PmwChWv1bky83vNIbS3xXnnB8iWcrMkTX2tCjtXaFI8

Workaround

$ export GIT_LFS_SKIP_SMUDGE=1
@pbrit
Copy link
Author

@pbrit pbrit commented May 8, 2020

Public dependencies with Git LFS files might be affected as well.

@toothrot toothrot changed the title checksum mismatch with private dependent module with GIT LFS files cmd/go: checksum mismatch with private dependent module with GIT LFS files May 11, 2020
@toothrot toothrot added this to the Backlog milestone May 11, 2020
@toothrot
Copy link
Contributor

@toothrot toothrot commented May 11, 2020

@bcmills
Copy link
Member

@bcmills bcmills commented May 11, 2020

What did you do?

$ go run main.go

Since you have obtained two different checksums, it seems likely that the repo has been cloned using two different configurations: one when the module was fetched initially (creating the go.sum entry), and a different one when the module was re-fetched (creating the conflicting checksum).

The “What did you do?” step needs to describe both of those configurations, not just the most recent one. (Otherwise, how can we reproduce the failure to ensure that it is fixed?)

@bcmills
Copy link
Member

@bcmills bcmills commented May 11, 2020

The go command uses git archive under the hood.

According to git-lfs/git-lfs#1322 (comment),

If you have Git LFS enabled (i.e., the filter rules are properly set up via git lfs install), a recent version of git archive will include the LFS files in it, even in a bare repository.

So it's not obvious to me why GIT_LFS_SKIP_SMUDGE=1 would be the right resolution here: why would git-lfs users expect Go modules to include (or omit) LFS files, and under what conditions? Does this behavior vary with git and git-lfs versions, and is upgrading to a more recent git and/or git-lfs binary a viable workaround?

@gopherbot
Copy link

@gopherbot gopherbot commented Jun 11, 2020

Timed out in state WaitingForInfo. Closing.

(I am just a bot, though. Please speak up if this is a mistake or you have the requested information.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.