go/internal/srcimporter: failure in TestImportStdLib and TestImportCgo with exit status 1 on linux-s390x-ibm

[bcmills]

--- FAIL: TestImportStdLib (2.14s)
    srcimporter_test.go:36: import "archive/tar" failed (type-checking package "archive/tar" failed (/data/golang/workdir/go/src/archive/tar/stat_unix.go:11:2: could not import os/user (exit status 1)))
    srcimporter_test.go:40: import "archive/zip": 44.571294ms
    srcimporter_test.go:40: import "bufio": 357.155µs
    srcimporter_test.go:40: import "bytes": 472.582µs
    srcimporter_test.go:40: import "cmd/addr2line": 361.821005ms
    srcimporter_test.go:40: import "cmd/api": 220.883209ms
    srcimporter_test.go:40: import "cmd/asm/internal/arch": 255.379371ms
    srcimporter_test.go:40: import "cmd/asm/internal/asm": 25.790491ms
    srcimporter_test.go:40: import "cmd/asm/internal/flags": 67.964µs
    srcimporter_test.go:40: import "cmd/asm/internal/lex": 193.852µs
    srcimporter_test.go:40: import "cmd/asm": 5.145132ms
    srcimporter_test.go:40: import "cmd/buildid": 15.045351ms
    srcimporter_test.go:40: import "cmd/cgo": 51.580322ms
    srcimporter_test.go:40: import "cmd/compile/internal/abi": 72.605405ms
    srcimporter_test.go:40: import "cmd/compile/internal/amd64": 390.41801ms
    srcimporter_test.go:40: import "cmd/compile/internal/arm": 12.544434ms
    srcimporter_test.go:40: import "cmd/compile/internal/arm64": 11.0452ms
    srcimporter_test.go:40: import "cmd/compile/internal/base": 272.965µs
    srcimporter_test.go:40: import "cmd/compile/internal/bitvec": 56.585µs
    srcimporter_test.go:40: import "cmd/compile/internal/deadcode": 1.86738ms
    srcimporter_test.go:40: import "cmd/compile/internal/devirtualize": 2.402334ms
    srcimporter_test.go:40: import "cmd/compile/internal/dwarfgen": 9.907358ms
    srcimporter_test.go:40: import "cmd/compile/internal/escape": 360.027µs
    srcimporter_test.go:40: import "cmd/compile/internal/gc": 221.270011ms
    srcimporter_test.go:49: testing time used up
    srcimporter_test.go:95: tested 24 imports
--- FAIL: TestCgo (0.09s)
    srcimporter_test.go:249: Import failed: exit status 1
FAIL
FAIL	go/internal/srcimporter	2.310s

greplogs --dashboard -md -l -e 'FAIL: Test(?:ImportStdLib|Cgo) .*(?:\n .*)exit status 1' --since=2022-01-01 --omit=.\*-n2d

2022-04-17T01:33:13-740a490/linux-s390x-ibm

[bcmills]

At the very least, that error message needs to be improved to explain what exited with exit status 1.

The type checking package … failed error appears to come from here:
https://cs.opensource.google/go/go/+/master:src/go/internal/srcimporter/srcimporter.go;l=154;drc=c0158b6a00eaecbd28ded0f66e65b9985f6db078

That implies that (*go/types.Config).Check) returned the error in question.
Tracing it down a few more steps:
https://cs.opensource.google/go/go/+/master:src/go/types/api.go;l=411;drc=master
https://cs.opensource.google/go/go/+/master:src/go/types/check.go;l=308;drc=master
https://cs.opensource.google/go/go/+/master:src/go/types/check.go;l=317;drc=master
...and now the origin of the error is obfuscated by the use of panic and recover for error-handling. 😩

[bcmills]

OTOH, I don't see any use of os/exec in go/types or its dependencies.
That suggests that the error is coming from the (*srcimporter).Importer itself, which is injected into the type-checker here:
https://cs.opensource.google/go/go/+/master:src/go/internal/srcimporter/srcimporter.go;l=127;drc=c0158b6a00eaecbd28ded0f66e65b9985f6db078

And, indeed, it has the potential to return a raw exec.ExitError in at least two places:
https://cs.opensource.google/go/go/+/master:src/go/internal/srcimporter/srcimporter.go;l=228;drc=master
https://cs.opensource.google/go/go/+/master:src/go/internal/srcimporter/srcimporter.go;l=240;drc=master

[bcmills]

I suspect that this particular failure mode is due to a platform or cgo bug on the linux-s390x builder (#52408), but we ought to fix the srcimporter error messages either way.

[gopherbot]

Change https://go.dev/cl/400817 mentions this issue: go/internal/srcimporter: add context to cgo errors

[jonathan-albrecht-ibm]

I should have closed this earlier. It was the same issue as #52408. Reinstalling the gcc-c++ and related packages at the correct versions on the builder fixed this issue back on Apr 18.