cmd/go: "go build" indefinitely stuck due to invoking git

[NuLL3rr0r]

What version of Go are you using (go version)?

$ go version
go version go1.20.2 linux/amd64

Does this issue reproduce with the latest release?

Yes!

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/mamadou/.cache/go-build"
GOENV="/home/mamadou/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/mamadou/dev/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/mamadou/dev/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.20.2"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="x86_64-pc-linux-gnu-gcc"
CXX="x86_64-pc-linux-gnu-g++"
CGO_ENABLED="1"
GOMOD="/home/mamadou/dev/MamadouArchives/MamadouArchives/Source/ThirdParty/MamadouArchivesBackend/go.mod"
GOWORK=""
CGO_CFLAGS="-O2 -g"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-O2 -g"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build280206031=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Well, I have a heavy Git/LFS repository hosting Unreal Engine + a Game project that relies on some utilities written in Golang. It was fine until I created another branch with the latest version of the engine and merged that to the master branch. Now, the issue is not Unreal Engine or Git, but go build. When I invoke go build command from my Makefile it gets stuck forever. Then I tried to track down the issue, so in that go module I invoke:

$ go build -x
WORK=/tmp/go-build3881985498
cd /home/mamadou/dev/MamadouArchives
git status --porcelain
cd /home/mamadou/dev/MamadouArchives
git -c log.showsignature=false show -s --format=%H:%ct

It's been stuck for 10 minutes now and never finished. Then, I onlly copy the Go module anywhere outside the Git repo and go build -x produces the binary just fine.

What did you expect to see?

I guess it should not be stuck. A timeout? Maybe! Though, I will try to leave it over night to see if it ever finishes.

What did you see instead?

Nothing! It just hangs forever.

[cherrymui]

cc @bcmills @matloob

[cherrymui]

@NuLL3rr0r does this reproduce with Go 1.19 or other version of Go? Thanks.

[NuLL3rr0r]

I don't have any other versions installed. Though, I left the PC for around half an hour and when I came back it was done as expected. Probably Git has done some background work. Now, when I invoke make it just builds as usual. But, I guess this is still a valid issue for huge repositories like mine, and I'd like to be able to at least provide a timeout for such a operations, e.g. through an env var. Though, feel free to close it if you don't think so.

[seankhliao]

since you're saying git-lfs, I guess it's downloading things into the module cache
so subsequent runs are faster.

you could use go build -x with a clean (or temporary) module cache to see what's sloe

[NuLL3rr0r]

The repo did not download anything. I am using i3wm and always have my dl/up speed at the top bar and I actually had an eye on that to see if it's downloading. Not to mention I am the sole user of this repo and there was no new commits.

As I said, I am suspecting the culprit might be Git housekeeping. Also, my filesystem is ZFS if it makes a difference that might as well do some housekeeping.

When originally, I merged the new branch it took around 14/15 hours to merge the updated branch to master. Unreal Engine is huge just checkout their repo. I guess I had just one commit with over 160K files modified.

My point is it's good to have a timeout. I might be wrong and I am the only one who keep their golang tools in the same repo as the whole engine and the game. But, this is common practice at a few game studios I have worked for as well. Though the ususally use SVN or Perforce.