cmd/go: 'go list -export' doesn't report export data file path when GOROOT is wrong

[matloob]

cc @jayconrod

$ go version
go version go1.12 darwin/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/matloob/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/matloob/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/matloob/tools/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/f4/9p58ddnj40x58zchdb36p7lr004_sl/T/go-build504723835=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

I did a go list with a GOROOT set to a different Go installations's goroot.
I have Go 1.12 installed in /usr/local/go/bin/go, and something tip-ish installed in $HOME/devgo

if I do a normal go list -export of a stdlib package, exports show up fine:

matloob-macbookpro2:tools matloob$ /usr/local/go/bin/go list -export -f "{{.Export}}" fmt
/usr/local/go/pkg/darwin_amd64/fmt.a

On the other hand running go list with GOROOT set to a different go installation's goroot, the export data is missing:

matloob-macbookpro2:tools matloob$ GOROOT=$HOME/devgo /usr/local/go/bin/go list -export -e -json -f "{{.Export}}" fmt
# math/bits
compile: version "devel +6d1aaf143c Tue Jun 25 21:47:04 2019 +0000" does not match go tool version "go1.12"
# unicode/utf8
compile: version "devel +6d1aaf143c Tue Jun 25 21:47:04 2019 +0000" does not match go tool version "go1.12"
# internal/race
compile: version "devel +6d1aaf143c Tue Jun 25 21:47:04 2019 +0000" does not match go tool version "go1.12"
# runtime/internal/sys
compile: version "devel +6d1aaf143c Tue Jun 25 21:47:04 2019 +0000" does not match go tool version "go1.12"
# unicode
compile: version "devel +6d1aaf143c Tue Jun 25 21:47:04 2019 +0000" does not match go tool version "go1.12"
# sync/atomic
compile: version "devel +6d1aaf143c Tue Jun 25 21:47:04 2019 +0000" does not match go tool version "go1.12"
# internal/cpu
compile: version "devel +6d1aaf143c Tue Jun 25 21:47:04 2019 +0000" does not match go tool version "go1.12"
# runtime/internal/atomic
compile: version "devel +6d1aaf143c Tue Jun 25 21:47:04 2019 +0000" does not match go tool version "go1.12"
matloob-macbookpro2:tools matloob$ echo $?
2

And the exit status is 2 so at least we know something's wrong.

But it's worse if we do a list -e. The same "warnings" are printed, but the exit status is 0 and an error isn't set on the package

matloob-macbookpro2:tools matloob$ GOROOT=$HOME/devgo /usr/local/go/bin/go list -export -e -json fmt
# internal/race
compile: version "devel +6d1aaf143c Tue Jun 25 21:47:04 2019 +0000" does not match go tool version "go1.12"
# math/bits
compile: version "devel +6d1aaf143c Tue Jun 25 21:47:04 2019 +0000" does not match go tool version "go1.12"
# unicode/utf8
compile: version "devel +6d1aaf143c Tue Jun 25 21:47:04 2019 +0000" does not match go tool version "go1.12"
# unicode
compile: version "devel +6d1aaf143c Tue Jun 25 21:47:04 2019 +0000" does not match go tool version "go1.12"
# runtime/internal/sys
compile: version "devel +6d1aaf143c Tue Jun 25 21:47:04 2019 +0000" does not match go tool version "go1.12"
# sync/atomic
compile: version "devel +6d1aaf143c Tue Jun 25 21:47:04 2019 +0000" does not match go tool version "go1.12"
# internal/cpu
compile: version "devel +6d1aaf143c Tue Jun 25 21:47:04 2019 +0000" does not match go tool version "go1.12"
# runtime/internal/atomic
compile: version "devel +6d1aaf143c Tue Jun 25 21:47:04 2019 +0000" does not match go tool version "go1.12"
{
	"Dir": "/Users/matloob/devgo/src/fmt",
	"ImportPath": "fmt",
	"Name": "fmt",
	"Doc": "Package fmt implements formatted I/O with functions analogous to C's printf and scanf.",
	"Target": "/Users/matloob/devgo/pkg/darwin_amd64/fmt.a",
	"Root": "/Users/matloob/devgo",
	"Match": [
		"fmt"
	],
	"Goroot": true,
	"Standard": true,
	"GoFiles": [
		"doc.go",
		"errors.go",
		"format.go",
		"print.go",
		"scan.go"
	],
	"Imports": [
		"errors",
		"internal/fmtsort",
		"io",
		"math",
		"os",
		"reflect",
		"strconv",
		"sync",
		"unicode/utf8"
	],
	"Deps": [
		"errors",
		"internal/bytealg",
		"internal/cpu",
		"internal/fmtsort",
		"internal/oserror",
		"internal/poll",
		"internal/race",
		"internal/reflectlite",
		"internal/syscall/unix",
		"internal/testlog",
		"io",
		"math",
		"math/bits",
		"os",
		"reflect",
		"runtime",
		"runtime/internal/atomic",
		"runtime/internal/math",
		"runtime/internal/sys",
		"sort",
		"strconv",
		"sync",
		"sync/atomic",
		"syscall",
		"time",
		"unicode",
		"unicode/utf8",
		"unsafe"
	],
	"TestGoFiles": [
		"export_test.go"
	],
	"XTestGoFiles": [
		"errors_test.go",
		"example_test.go",
		"fmt_test.go",
		"gostringer_example_test.go",
		"scan_test.go",
		"stringer_example_test.go",
		"stringer_test.go"
	],
	"XTestImports": [
		"bufio",
		"bytes",
		"errors",
		"fmt",
		"internal/race",
		"io",
		"math",
		"os",
		"reflect",
		"regexp",
		"runtime",
		"strings",
		"testing",
		"testing/iotest",
		"time",
		"unicode",
		"unicode/utf8"
	]
}

What did you expect to see?

The ideal situation is that the value GOROOT is ignored. The go command already knows its own GOROOT, and using a different one will break things.

At the least, if go list is going to fail, it should have an error set.

What did you see instead?

go list -e -json -export succeeds, but has no export data present.

[jayconrod]

It looks like these errors are coming from the compiler.

The Go command wants to build these packages at its own version (devel +6d1aaf143c Tue Jun 25 21:47:04 2019 +0000). It invokes the compiler from GOTOOLDIR, which is derived from the explicitly set GOROOT. The compiler was built with go1.12, and that doesn't match the requested version. It can only build for the same version.

I don't think it makes sense to ignore GOROOT entirely. This would work if the Go command were built with the same version of Go as GOROOT. That's probably necessary for bootstrapping, though I'm not sure exactly how. I expect some folks have a workflow that takes advantage of this, too.

It's definitely a problem that the errors aren't reported correctly in the JSON when the -e flag is set though. If the Go command runs a tool that exits non-zero, the Go command assumes the tool printed something and sets its own exit status to 2 (which is what's happening here). For go list -e specifically, we should capture stderr and push it into the Error field of the Package that corresponds to the action.