Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: "go list -m all" hangs for git.apache.org #34016

Closed
ukai opened this issue Sep 2, 2019 · 11 comments
Closed

cmd/go: "go list -m all" hangs for git.apache.org #34016

ukai opened this issue Sep 2, 2019 · 11 comments

Comments

@ukai
Copy link
Contributor

@ukai ukai commented Sep 2, 2019

What version of Go are you using (go version)?

$ go version
go version go1.12.8 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/gopath/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/gopath"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/workspace/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build417902936=/tmp/go-build -gno-record-gcc-switches"

What did you do?

with warmed go module cache, it is not problem.

It doesn't require git.apache.org/thrift.git module.
e.g. go list -m all doesn't show git.apache.org/thrift.git nor go mod why git.apache.org/thrift.git show 'not need package'.

in go.sum,

git.apache.org/thrift.git v0.0.0-20180902110319-2566ecd5d999/go.mod h1:fPE2ZNJGynbRyZ4dJvy6G277gSllfV2HJqblrnkyeyg=

was written.
(probably go.opencensus.io v0.18.0?)

go mod tidy won't clear this entry.

What did you expect to see?

go list -m all didn't hang.

What did you see instead?

go list -m all hangs to try to check git.apache.org/thrift.git
if used with new go module cache.

adding replace git.apache.org/thrift.git => github.com/apache/thrift v0.12.0 could fix this.

@dcelasun

This comment has been minimized.

Copy link

@dcelasun dcelasun commented Sep 2, 2019

Thrift has officially switched to to github with v0.12.0, which was released last year. We've supported cloning from git.apache.org for a while for backwards compatibility, but that's gone now. You don't need a replace directive, just import github.com/apache/thrift.

@mfmayer

This comment has been minimized.

Copy link

@mfmayer mfmayer commented Sep 2, 2019

The problem still persist if other required modules still require git.apache.org/thrift.git. In that case the error is very persistent and I had some difficulties to fix it. Even with the replace statement, I first couldn’t get rid of it. It was even difficult to find out which modules still require the old URL, because go mod graph only complaints about problems with git.apache.org as well as go mod tidy and even go clean —modcache.

I can’t tell anymore but I believe deleting the modules cache manually finally did the trick (while keeping the replace statement).

@ukai

This comment has been minimized.

Copy link
Contributor Author

@ukai ukai commented Sep 3, 2019

thanks, go mod graph tells which modules still requires the old module.

@ALTree ALTree changed the title `go list -m all` hangs for git.apache.org cmd/go: "go list -m all" hangs for git.apache.org Sep 3, 2019
@ALTree ALTree added this to the Go1.14 milestone Sep 3, 2019
@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Sep 3, 2019

I'm not able to reproduce a hang. I do see a long delay during the direct-mode fetch, but it eventually resolves to a useful error message.

@ukai, is there something in your global Git config that would prevent git commands from timing out as usual?

Both runs starting from a clean module cache:

example.com$ gotip mod init example.com
go: creating new go.mod: module example.com

example.com$ cat | gofmt > main.go
package main

import _ "git.apache.org/thrift.git"

func main() {}

example.com$ gotip list -m all
example.com

example.com$ gotip mod tidy
go: finding git.apache.org/thrift.git v0.12.0
go: downloading git.apache.org/thrift.git v0.12.0
go: extracting git.apache.org/thrift.git v0.12.0
example.com imports
        git.apache.org/thrift.git: module git.apache.org/thrift.git@latest (v0.12.0) found, but does not contain package git.apache.org/thrift.git
example.com$ gotip mod init example.com
go: creating new go.mod: module example.com

example.com$ cat | gofmt > main.go
package main

import _ "git.apache.org/thrift.git"

func main() {}

example.com$ GOPROXY=direct gotip mod tidy
example.com imports
        git.apache.org/thrift.git: git ls-remote -q origin in /tmp/tmp.QfnCyUFnCW/_gopath/pkg/mod/cache/vcs/f6f367fb9d5656665ab3c2c1c3b2a44c2ffc79f6b70612f5864d7ceedcc7b9ea: exit status 128:
        fatal: unable to access 'https://git.apache.org/thrift.git/': Failed to connect to git.apache.org port 443: Connection timed out
@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Sep 3, 2019

Change https://golang.org/cl/193039 mentions this issue: all: upgrade dependencies on google.golang.org/api and go.opencensus.io

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Sep 3, 2019

Change https://golang.org/cl/193097 mentions this issue: blog: add replace directives for git.apache.org

gopherbot pushed a commit to golang/website that referenced this issue Sep 3, 2019
The older versions impose transitive dependencies on
git.apache.org/thrift.git, which has since been removed.

Commands run:

	go get -d google.golang.org/api go.opencensus.io
	go mod tidy
	go test ./...

Updates golang/go#34016

Change-Id: I32f4926cac55d630febd9d4a4412eeb99ed9878e
Reviewed-on: https://go-review.googlesource.com/c/website/+/193039
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Bonventre <andybons@golang.org>
@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Sep 3, 2019

Change https://golang.org/cl/193098 mentions this issue: all: upgrade dependency on x/website

gopherbot pushed a commit to golang/blog that referenced this issue Sep 3, 2019
This removes the transitive dependency on git.apache.org/thrift.git,
dropped from x/website in CL 193039.

Commands run:

	go get -d golang.org/x/website@00e97558e113808bd2e87edf2275d97515bad94b
	go mod tidy
	go test ./...

Updates golang/go#34016

Change-Id: I903f1eb29454db1c76945bea97a77d5748913c8c
Reviewed-on: https://go-review.googlesource.com/c/blog/+/193098
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Bonventre <andybons@golang.org>
@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Sep 3, 2019

As far as I can tell there is nothing more to be done here. As @ukai notes, you can use a replace directive as a temporary workaround, and as @dcelasun notes, it is usually possible to remove the bad dependency by upgrading past whatever depends on it.

You can figure out “whatever depends on it” using go mod graph | grep git.apache.org.


For the replace directive, I would recommend replacing the specific commit with the same specific commit under github.com, since that will be least disruptive to the rest of the module graph. For example:

replace git.apache.org/thrift.git v0.0.0-20181218151757-9b75e4fe745a => github.com/apache/thrift v0.0.0-20181218151757-9b75e4fe745a
@dcelasun

This comment has been minimized.

Copy link

@dcelasun dcelasun commented Sep 3, 2019

Also, I plan on including a go.mod file with the upcoming v0.13.0 of thrift, so pinning a specific commit shouldn't be necessary after that.

@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Sep 3, 2019

@dcelasun, adding a go.mod file in the github.com/apache/thrift repo seems fine, but won't really help with this issue in particular.

(github.com/apache/thrift/lib/go/thrift already resolves just fine at v0.12.0 — it's the change in import path that broke things, not the lack of go.mod file.)

@dcelasun

This comment has been minimized.

Copy link

@dcelasun dcelasun commented Sep 3, 2019

@bcmills sorry my comment was a bit tangential, just wanted to mention it after seeing that replace line.

@bcmills bcmills closed this Sep 3, 2019
sthaha added a commit to sthaha/tektoncd-pipeline-operator that referenced this issue Oct 3, 2019
golangci-lint breaks because go list fails due to `thrift` package
issue; see: golang/go#34016 and later due to
another issue golangci/golangci-lint#658 for
which a release of golangci-lint is yet to be made.
This patch fixes these by moving to operator-sdk version 0.10 and by
installing (go get) latest golangci-lint

Signed-off-by: Sunil Thaha <sthaha@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.