cmd/go: GOROOT in hidden directory, didn't find the well-known module "std"

[p-kraszewski]

What version of Go are you using (go version)?

$ go version
go version go1.18.4 freebsd/amd64

Does this issue reproduce with the latest release?

Yes, on 2 different operating systems

• FreeBSD 13.1 with Go 1.18.4 from packages
• Arch Linux with Go 1.19 from packages - the error messages are exactly the same, didn't copy output twice.

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="freebsd"
GOINSECURE=""
GOMODCACHE="/root/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="freebsd"
GOPATH="/root/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go118"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go118/pkg/tool/freebsd_amd64"
GOVCS=""
GOVERSION="go1.18.4"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="cc"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build3287959224=/tmp/go-build -gno-record-gcc-switches"

What did you do?

> git describe --tags --long --all
tags/go1.19-0-g43456202a1

> ./all.bash

What did you expect to see?

Building Go cmd/dist using /usr/local/go118. (go1.18.4 freebsd/amd64)
Building Go toolchain1 using /usr/local/go118.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
Building Go toolchain3 using go_bootstrap and Go toolchain2.
Building packages and commands for freebsd/amd64.

##### Test execution environment.
# GOARCH: amd64
# CPU: Intel(R) Xeon(R) CPU E5-2690 v3 @ 2.60GHz
# GOOS: freebsd
# OS Version: FreeBSD 13.1-RELEASE FreeBSD 13.1-RELEASE releng/13.1-n250148-fc952ac2212 MAXPHYS  amd64

##### Testing packages.
ok      archive/tar     0.461s
ok      archive/zip     0.596s
ok      bufio   0.331s
ok      bytes   0.284s
ok      compress/bzip2  0.764s
[...]
All tests successful

What did you see instead?

Building Go cmd/dist using /usr/local/go118. (go1.18.4 freebsd/amd64)
Building Go toolchain1 using /usr/local/go118.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
Building Go toolchain3 using go_bootstrap and Go toolchain2.
Building packages and commands for freebsd/amd64.

##### Test execution environment.
# GOARCH: amd64
# CPU: Intel(R) Xeon(R) CPU E5-2690 v3 @ 2.60GHz
# GOOS: freebsd
# OS Version: FreeBSD 13.1-RELEASE FreeBSD 13.1-RELEASE releng/13.1-n250148-fc952ac2212 MAXPHYS  amd64

##### Testing packages.
ok      archive/tar     0.461s
ok      archive/zip     0.596s
ok      bufio   0.331s
ok      bytes   0.284s
ok      compress/bzip2  0.764s

[...]

ok      cmd/internal/goobj      0.045s
--- FAIL: TestAllDependencies (0.00s)
moddeps_test.go:49: findGorootModules didn't find the well-known module "std"
--- FAIL: TestDependencyVersionsConsistent (0.00s)
moddeps_test.go:350: findGorootModules didn't find the well-known module "std"
FAIL
FAIL    cmd/internal/moddeps    0.061s
ok      cmd/internal/notsha256  0.177s

[...]

ok      cmd/vet 7.485s
FAIL
go tool dist: Failed: exit status 1

[seankhliao]

please try this with the upstream source tree.

[p-kraszewski]

If by a upstream source tree you mean one from the official manual:

git clone https://go.googlesource.com/go goroot
cd goroot
git checkout release-branch.go1.19

# This today still equals to tag go1.19

then my source code was exactly that.

> git log

commit 43456202a1e55da55666fac9d56ace7654a65b64 (HEAD -> release-branch.go1.19, tag: go1.19, origin/release-branch.go1.19)
Author: Gopher Robot <gobot@golang.org>
Date:   Tue Aug 2 16:39:14 2022 +0000

[release-branch.go1.19] go1.19

Change-Id: Iea643b04a7eea7c882d7eec481b67e4ecc08a559
Reviewed-on: https://go-review.googlesource.com/c/go/+/420834
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>

On a fresh account (so only a system Go is visible for bootstrap) with abovementioned source checkout still gets the two errors.

[dmgk]

I cannot reproduce this on 14.0-CURRENT. Is there anything special in the environment?

[p-kraszewski]

Nothing special.

• OS: FreeBSD MaxBSD-1 13.1-RELEASE FreeBSD 13.1-RELEASE releng/13.1-n250148-fc952ac2212 MAXPHYS amd64 (That MAXPHYS is just two-line config tweak bumping up maximal DMA transfer size.)
• x86_64 machine (2x 12C/24T CPU, 256GB RAM)
• System GO (1st phase bootstrap): go version go1.18.4 freebsd/amd64 installed by pkg install go
• No GO environment variables set (GOROOT, GOPATH, etc)
• Shell: ZSH with starship, but the error is reproducible from vanilla csh.
• No fancy environment settings (csh has unaltered after-the-install vanilla settings).

Nothing unusual and exactly reproducible on Arch machine (other OS, other bootstrap version, other hardware).

The only thing in common is that neither of machines is compiling go from ${HOME}/go - but the example in the manual is neither. My regular compilations run from ${HOME}/.go and experiments on a clean account were from ${HOME}/.goroot.

[p-kraszewski]

Got it!

The cmd/internal/moddeps test fails, when the directory with go source is hidden!

[seankhliao]

cc @bcmills @matloob