Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: unable to "go get" repository with external LFS server (go modules) #39720

Open
campbellr opened this issue Jun 19, 2020 · 5 comments
Open

Comments

@campbellr
Copy link

@campbellr campbellr commented Jun 19, 2020

What version of Go are you using (go version)?

$ go version
go version go1.14.3 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/ryancampbell/Library/Caches/go-build"
GOENV="/Users/ryancampbell/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GONOPROXY="github.com/kirasystems/*"
GONOSUMDB="github.com/kirasystems/*"
GOOS="darwin"
GOPATH="/Users/ryancampbell/go"
GOPRIVATE="github.com/kirasystems/*"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.14.3/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.14.3/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/k7/np6wbqs53wvgc5167kfc81yr0000gp/T/go-build056608014=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I have a repository that use Git LFS, with a external LFS server configured via a .lfsconfig file in the root of the repository (docs).

When I attempt to go get a package in this repo in "module-aware mode", it fails with the following error:

$ go get github.com/campbellr/goproject/package
go: downloading github.com/campbellr/goproject v0.0.0-20200618225840-1c974a2e2163
go: downloading github.com/campbellr/goproject/package v0.0.0-20200618225840-1c974a2e2163
go get github.com/campbellr/goproject/package: git -c core.autocrlf=input -c core.eol=lf archive --format=zip --prefix=prefix/ 1c974a2e2163f0097eddb66edd91ebbb725cc3fc -- package in /Users/ryancampbell/go/pkg/mod/cache/vcs/bc6786f5d692de19e81d93e06b7c197cfd764a6b5981aeb750293aad946567ca: exit status 128:
        Downloading package/some-lfs-file.model (9.8 KB)
        Error downloading object: package/some-lfs-file.model (0759275): Smudge error: Error downloading package/some-lfs-file.model (0759275b587490b6e6124f421b9781362517cd1aa2354c4e849f3949adb05f94): [0759275b587490b6e6124f421b9781362517cd1aa2354c4e849f3949adb05f94] Object does not exist on the server: [404] Object does not exist on the server

        Errors logged to /Users/ryancampbell/go/pkg/mod/cache/vcs/bc6786f5d692de19e81d93e06b7c197cfd764a6b5981aeb750293aad946567ca/lfs/logs/20200619T084543.526724.log
        Use `git lfs logs last` to view the log.
        error: external filter 'git-lfs filter-process' failed
        fatal: package/some-lfs-file.model: smudge filter lfs failed

As far as I can tell, the problem seems to be due to the way that go get clones the git repository.

  1. git init --bare
  2. git remote add origin github.com/campbellr/goproject
  3. git fetch
  4. git archive thepackage/ directory

Since git archive is running in a bare repository, there is no .lfsconfig, so git-lfs doesn't fetch the objects from the right lfs server and gets a 404.

Note that the same go get will succeed in GOPATH mode (since it just does a normal git clone)

@cagedmantis cagedmantis changed the title cmd/go: Unable to "go get" repository with external LFS server (go modules) cmd/go: unable to "go get" repository with external LFS server (go modules) Jun 22, 2020
@cagedmantis cagedmantis added this to the Unplanned milestone Jun 22, 2020
@cagedmantis
Copy link
Contributor

@cagedmantis cagedmantis commented Jun 22, 2020

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Jun 23, 2020

Related #25605, especially this comment.

For posterity: I think that any repositories being built by vgo depend on a Git LFS object during compilation will fail, since those objects will not be checked out, (because the repository is bare). That's a limitation of git-archive(1) and Git LFS, but not vgo.

@campbellr
Copy link
Author

@campbellr campbellr commented Jun 23, 2020

tbh, I wasn't sure if this was more appropriate as a git-lfs or git-archive issue rather than a Go issue, but I ended up filing it here because:

  1. This is a regression from GOPATH mode go get, which works as expected
  2. This is a result of go get cloning the repo in an "unexpected" way. I'm sure there is a good reason for it, but it does seem break assumptions that git-lfs (and possibly other tools?) seems to depend on.
@bcmills
Copy link
Member

@bcmills bcmills commented Jun 23, 2020

See also #29987 (also related to LFS).

@bcmills
Copy link
Member

@bcmills bcmills commented Jun 23, 2020

See also #38941 (comment) (somehow related to git-lfs smudging).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.