Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: "fatal: git fetch-pack: expected shallow list" when retrieving earlier version of already installed package on CentOS 7. #38373

Open
dhowell66 opened this issue Apr 11, 2020 · 12 comments

Comments

@dhowell66
Copy link

@dhowell66 dhowell66 commented Apr 11, 2020

What version of Go are you using (go version)?

$ go version
go version go1.13.6 linux/amd64

Does this issue reproduce with the latest release?

Not sure, but this is the latest supported version for CentOS 7.

What operating system and processor architecture are you using (go env)?

CentOS 7.

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/users/d00u3151/.cache/go-build"
GOENV="/users/d00u3151/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/users/d00u3151/go"
GOPRIVATE=""
GOPROXY="direct"
GOROOT="/usr/lib/golang"
GOSUMDB="off"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/golang/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/users/d00u3151/tmp/go-build945266341=/tmp/go-build -gno-record-gcc-switches"
[d00u3151@pde-bsp3hq2 dan]$

What did you do?

Tried to build the latest unreleased version of reposurgeon (problem does not occur with reposurgeon release 4.5).

$ git clone https://gitlab.com/esr/reposurgeon.git
Cloning into 'reposurgeon'...
remote: Enumerating objects: 1203, done.
remote: Counting objects: 100% (1203/1203), done.
remote: Compressing objects: 100% (456/456), done.
remote: Total 33416 (delta 890), reused 1009 (delta 739), pack-reused 32213
Receiving objects: 100% (33416/33416), 550.06 MiB | 11.28 MiB/s, done.
Resolving deltas: 100% (22980/22980), done.
$ cd reposurgeon
$ git checkout -q b392d176f30825d150b828c4df3d2d8e2b3166f7 # version as of April 10, 2020
$ make
asciidoctor -a nofooter -b manpage reposurgeon.adoc
asciidoctor -a nofooter -b manpage repocutter.adoc
asciidoctor -a nofooter -b manpage repomapper.adoc
asciidoctor -a nofooter -b manpage repotool.adoc
asciidoctor -a nofooter -b manpage repobench.adoc
asciidoctor -a webfonts! reposurgeon.adoc
asciidoctor -a webfonts! repocutter.adoc
asciidoctor -a webfonts! repomapper.adoc
asciidoctor -a webfonts! repotool.adoc
asciidoctor -a webfonts! repobench.adoc
asciidoctor -a webfonts! repository-editing.adoc
go build -gcflags '-N -l' -o repocutter ./cutter
go: github.com/go-delve/delve@v1.4.0 requires
        github.com/sirupsen/logrus@v0.0.0-20180523074243-ea8897e79973: invalid version: git fetch --unshallow -f origin in /users/d00u3151/go/pkg/mod/cache/vcs/020616345f7c7f88438c217f9d0e26744bce721c80e4a28f93399a8a4cd2acf1: exit status 128:
        fatal: git fetch-pack: expected shallow list
make: *** [build] Error 1
$ make
go build -gcflags '-N -l' -o repocutter ./cutter
go: github.com/go-delve/delve@v1.4.0 requires
        golang.org/x/arch@v0.0.0-20190927153633-4e8777c89be4: invalid version: git fetch --unshallow -f https://go.googlesource.com/arch in /users/d00u3151/go/pkg/mod/cache/vcs/a260a67b53c91dca287c34ff2bdee130be447c2ea411a64bf4489a69e886411b: exit status 128:
        fatal: git fetch-pack: expected shallow list
make: *** [build] Error 1

What did you expect to see?

A successful build.

What did you see instead?

A cryptic error message.

This seems to occur when multiple versions of the same package are required. In this case, the go.mod contains golang.org/x/arch v0.0.0-20200312215426-ff8b605520f4 and github.com/go-delve/delve v1.4.0, but delve v1.4.0 has a dependency on: golang.org/x/arch@v0.0.0-20190927153633-4e8777c89be4. It seems that trying to get the earlier version of arch when the later version has already been retrieved causes this error message.

In this case I was able to work around and successfully build by removing the reference to the later version of arch from go.mod.

@seankhliao
Copy link
Contributor

@seankhliao seankhliao commented Apr 11, 2020

this looks like the git command being too old to support shallow, solutions would be to upgrade git and/or enable the module proxy so git is no longer necessary

@dhowell66
Copy link
Author

@dhowell66 dhowell66 commented Apr 11, 2020

@dhowell66
Copy link
Author

@dhowell66 dhowell66 commented Apr 11, 2020

Strange thing is if I manually create a shallow clone, git fetch --unshallow works fine. I'm not sure what is different about the shallow clones that go retrieves vs. the following:

$ git version
git version 1.8.3.1
$ git clone --depth 1 --bare https://go.googlesource.com/arch
Cloning into bare repository 'arch.git'...
remote: Counting objects: 144, done
remote: Finding sources: 100% (144/144)
remote: Total 144 (delta 11), reused 129 (delta 11)
Receiving objects: 100% (144/144), 768.76 KiB | 0 bytes/s, done.
Resolving deltas: 100% (11/11), done.
$ cd arch.git
$ git log --all --oneline
ff8b605 vendor: delete
$ git fetch --unshallow -f https://go.googlesource.com/arch
remote: Total 614 (delta 228), reused 614 (delta 228)
Receiving objects: 100% (614/614), 1.47 MiB | 0 bytes/s, done.
Resolving deltas: 100% (228/228), done.
From https://go.googlesource.com/arch
 * branch            HEAD       -> FETCH_HEAD
$ git log --all --oneline
ff8b605 vendor: delete
69f17b2 vendor: migrate from govendor to go mod vendor
368ea8f x86asm: add support for FLDZ and FLDLN2
a0d8588 arm64/arm64asm: rename NOP instruction as NOOP in go syntax
4e8777c arm64/arm64gen: get system register readable and writeable attribute
1137aed arm64/arm64gen: adds a generator sysreggen.go
[...snip...]
@bcmills
Copy link
Member

@bcmills bcmills commented Apr 11, 2020

See also #26746 and #29114.

@bcmills
Copy link
Member

@bcmills bcmills commented Apr 11, 2020

Also, old git clients have a bug for which the cmd/go workaround (#34092) has probably not been backported. You may need to upgrade to Go 1.14.1 if you are affected.

@bcmills bcmills changed the title Error "fatal: git fetch-pack: expected shallow list" when retrieving earlier version of already installed package on CentOS 7. cmd/go: "fatal: git fetch-pack: expected shallow list" when retrieving earlier version of already installed package on CentOS 7. Apr 11, 2020
@bcmills bcmills added this to the Unplanned milestone Apr 11, 2020
@dhowell66
Copy link
Author

@dhowell66 dhowell66 commented Apr 11, 2020

FYI, previous issues mentioned problems on CentOS 5 & CentOS 6. CentOS 5 is no longer supported, and CentOS 6 is reaching end-of-life later this year, but CentOS 7 is being supported until 2024.

@dolph
Copy link

@dolph dolph commented Apr 29, 2020

I just ran into this issue on CentOS 7.8 (released September 24, 2019), with golang and git both installed via yum (from CentOS repos).

go version go1.13.6 linux/ppc64le
git version 1.8.3.1

I resolved this by uninstalling go 1.13.6 via yum and installing go 1.14.2 from https://golang.org/dl/

@nilfdev
Copy link

@nilfdev nilfdev commented May 1, 2020

Faced with the same issues go version and git version are the same,
after second run error is the next

env GO111MODULE=on go run build/ci.go install ./cmd/geth
build command-line-arguments: cannot load golang_org/x/crypto/chacha20poly1305: cannot find module providing package golang_org/x/crypto/chacha20poly1305
make: *** [geth] Error 1

@dfedorov-ciena
Copy link

@dfedorov-ciena dfedorov-ciena commented May 27, 2020

Same issue here, same solution: remove EPEL's go 1.13 and install 1.14 directly from the vendor:

yum remove golang-bin
wget https://dl.google.com/go/go1.14.3.linux-amd64.tar.gz
etc.

@siscia
Copy link

@siscia siscia commented Jul 9, 2020

Same issue!

Unfortunately old platforms (and BTW Centos7 is not THAT old) are still vital in enterprise environment where stability is required, the suggestion to update git is a valid one but it really hamper down the velocity and ergonomics of go.

@jgallucci32
Copy link

@jgallucci32 jgallucci32 commented Oct 1, 2020

We have had similar issues running Bitbucket on RHEL7 in production needing a later version of the Git client. We have always found the IUS version to be very stable for production. Go to https://repo.ius.io/ to learn more.

Here are the basic steps to get the newer version of Git on Centos7/RHEL7:

yum -y erase git
yum -y install https://repo.ius.io/ius-release-el7.rpm
yum -y install git222

After updating to Git 2.2.2 the go get commands were successful.

@thompsbrian
Copy link

@thompsbrian thompsbrian commented Mar 2, 2021

I ran into the same error on OEL7 (RHEL-based) with git 2.24.0 and go 1.15.2.

The error I ran into resulted from the gh cli installer. I ran sudo make install initially, which resulted in the error.

To fix it, I then ran:

  1. make install (permission denied error)
  2. sudo make install again (finally successful)

Looks like it's possible related to sudo as well.

thaJeztah added a commit to thaJeztah/docker-ce-packaging that referenced this issue Mar 23, 2021
…HEL 7

Our Dockerfile uses GOPROXY=direct, which makes "go mod" commands use
git to fetch modules. "go mod" in Go versions before 1.14.1 are incompatible
with older git versions, including the version of git that ships with
CentOS/RHEL 7 (which have git 1.8), see golang/go#38373

This patch switches the scan plugin script to use the default GOPROXY so
that git is not required for downloading the modules.

Once all our code has upgraded to Go 1.14+, this workaround should be
removed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
thaJeztah added a commit to thaJeztah/docker-ce-packaging that referenced this issue Mar 23, 2021
…HEL 7

Our Dockerfile uses GOPROXY=direct, which makes "go mod" commands use
git to fetch modules. "go mod" in Go versions before 1.14.1 are incompatible
with older git versions, including the version of git that ships with
CentOS/RHEL 7 (which have git 1.8), see golang/go#38373

This patch switches the scan plugin script to use a proxy so that git is
not required for downloading the modules

Once all our code has upgraded to Go 1.14+, this workaround should be
removed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
tiborvass added a commit to tiborvass/docker that referenced this issue Mar 23, 2021
… CentOS/RHEL 7

Since rootlesskit removed vendor folder, building it has to rely on go mod.

Dockerfile in docker-ce-packaging uses GOPROXY=direct, which makes "go mod"
commands use git to fetch modules. "go mod" in Go versions before 1.14.1 are
incompatible with older git versions, including the version of git that ships
with CentOS/RHEL 7 (which have git 1.8), see golang/go#38373

This patch switches rootlesskit install script to set GOPROXY to
https://proxy.golang.org so that git is not required for downloading modules.

Once all our code has upgraded to Go 1.14+, this workaround should be
removed.

Signed-off-by: Tibor Vass <tibor@docker.com>
thaJeztah added a commit to thaJeztah/docker-ce-packaging that referenced this issue Mar 23, 2021
…HEL 7

Our Dockerfile uses GOPROXY=direct, which makes "go mod" commands use
git to fetch modules. "go mod" in Go versions before 1.14.1 are incompatible
with older git versions, including the version of git that ships with
CentOS/RHEL 7 (which have git 1.8), see golang/go#38373

This patch switches scan install script to set GOPROXY to
https://proxy.golang.org so that git is not required for downloading modules.

Once all our code has upgraded to Go 1.14+, this workaround should be
removed.

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
tiborvass added a commit to tiborvass/docker that referenced this issue Mar 23, 2021
… CentOS/RHEL 7

Since rootlesskit removed vendor folder, building it has to rely on go mod.

Dockerfile in docker-ce-packaging uses GOPROXY=direct, which makes "go mod"
commands use git to fetch modules. "go mod" in Go versions before 1.14.1 are
incompatible with older git versions, including the version of git that ships
with CentOS/RHEL 7 (which have git 1.8), see golang/go#38373

This patch switches rootlesskit install script to set GOPROXY to
https://proxy.golang.org so that git is not required for downloading modules.

Once all our code has upgraded to Go 1.14+, this workaround should be
removed.

Signed-off-by: Tibor Vass <tibor@docker.com>
thaJeztah added a commit to thaJeztah/docker that referenced this issue Mar 24, 2021
… CentOS/RHEL 7

Since rootlesskit removed vendor folder, building it has to rely on go mod.

Dockerfile in docker-ce-packaging uses GOPROXY=direct, which makes "go mod"
commands use git to fetch modules. "go mod" in Go versions before 1.14.1 are
incompatible with older git versions, including the version of git that ships
with CentOS/RHEL 7 (which have git 1.8), see golang/go#38373

This patch switches rootlesskit install script to set GOPROXY to
https://proxy.golang.org so that git is not required for downloading modules.

Once all our code has upgraded to Go 1.14+, this workaround should be
removed.

Signed-off-by: Tibor Vass <tibor@docker.com>
(cherry picked from commit cbc6cef)
Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
docker-jenkins pushed a commit to docker/docker-ce that referenced this issue Mar 24, 2021
… CentOS/RHEL 7

Since rootlesskit removed vendor folder, building it has to rely on go mod.

Dockerfile in docker-ce-packaging uses GOPROXY=direct, which makes "go mod"
commands use git to fetch modules. "go mod" in Go versions before 1.14.1 are
incompatible with older git versions, including the version of git that ships
with CentOS/RHEL 7 (which have git 1.8), see golang/go#38373

This patch switches rootlesskit install script to set GOPROXY to
https://proxy.golang.org so that git is not required for downloading modules.

Once all our code has upgraded to Go 1.14+, this workaround should be
removed.

Signed-off-by: Tibor Vass <tibor@docker.com>
Upstream-commit: cbc6cefdcbe8b7ab66571407ee0b3c39f1e7a96b
Component: engine
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
9 participants