cmd/go: recurses into directories only containing assembly .s files

[jdgordon]

What version of Go are you using (go version)?

$ go version
go version go1.19.1 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/jonathan/.cache/go-build"
GOENV="/home/jonathan/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/jonathan/go/pkg/mod"
GONOPROXY="gitlab.com/fleetspace"
GONOSUMDB="gitlab.com/fleetspace"
GOOS="linux"
GOPATH="/home/jonathan/go"
GOPRIVATE="gitlab.com/fleetspace"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/jonathan/go/go1.19.1"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/jonathan/go/go1.19.1/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.19.1"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/jonathan/git/golangci-lint/go.mod"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build872883275=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I originally filed this issue with golangci-lint however they are pushing back that this is a go tooling problem. golangci/golangci-lint#3339

The Issue is that after adding a git submodule of a completely unrelated project into any part of the tree under the go.mod, attempting to build gives the wonderful error as follows

 go build ./...
pattern ./...: main module (github.com/golangci/golangci-lint) does not contain package github.com/golangci/golangci-lint/loramac-node/src/boards/NucleoL152/cmsis/arm-std

I have reproduced this with a clean checkout of the golangci-lint repo, as well as creating a test repo with nothing but the git submodule. Both show the same behavior.

What did you expect to see?

no problems with go build or golangci-lint when running with random unrelated git tree in the workdir.

What did you see instead?

This is a copy of my terminal showing how to reproduce

jonathan@fleetlaptop:~$ cd git/
jonathan@fleetlaptop:~/git$ git clone https://github.com/golangci/golangci-lint.git
Cloning into 'golangci-lint'...
remote: Enumerating objects: 22754, done.
remote: Counting objects: 100% (24/24), done.
remote: Compressing objects: 100% (16/16), done.
remote: Total 22754 (delta 14), reused 17 (delta 8), pack-reused 22730
Receiving objects: 100% (22754/22754), 39.55 MiB | 5.99 MiB/s, done.
Resolving deltas: 100% (13907/13907), done.
jonathan@fleetlaptop:~/git$ cd golangci-lint/
jonathan@fleetlaptop:~/git/golangci-lint$ ~/go/go1.17.13/bin/go
go             gofmt          golangci-lint  
jonathan@fleetlaptop:~/git/golangci-lint$ ~/go/go1.17.13/bin/golangci-lint run
jonathan@fleetlaptop:~/git/golangci-lint$ git submodule add https://github.com/lora-net/loramac-node.git
Cloning into '/home/jonathan/git/golangci-lint/loramac-node'...
remote: Enumerating objects: 28497, done.
remote: Counting objects: 100% (321/321), done.
remote: Compressing objects: 100% (144/144), done.
remote: Total 28497 (delta 190), reused 266 (delta 171), pack-reused 28176
Receiving objects: 100% (28497/28497), 27.37 MiB | 6.29 MiB/s, done.
Resolving deltas: 100% (18361/18361), done.
jonathan@fleetlaptop:~/git/golangci-lint$ ~/go/go1.17.13/bin/golangci-lint run
ERRO [linters_context] typechecking error: pattern ./...: main module (github.com/golangci/golangci-lint) does not contain package github.com/golangci/golangci-lint/loramac-node/src/boards/NAMote72/cmsis/arm-std 
jonathan@fleetlaptop:~/git/golangci-lint$ git diff
jonathan@fleetlaptop:~/git/golangci-lint$ git diff --staged
diff --git a/.gitmodules b/.gitmodules
new file mode 100644
index 00000000..0aad7827
--- /dev/null
+++ b/.gitmodules
@@ -0,0 +1,3 @@
+[submodule "loramac-node"]
+       path = loramac-node
+       url = https://github.com/lora-net/loramac-node.git
diff --git a/loramac-node b/loramac-node
new file mode 160000
index 00000000..7f25f7ad
--- /dev/null
+++ b/loramac-node
@@ -0,0 +1 @@
+Subproject commit 7f25f7adb5b9bf7142f98f77c68363e8b1f7e8ee
jonathan@fleetlaptop:~/git/golangci-lint$ go build ./...
pattern ./...: main module (github.com/golangci/golangci-lint) does not contain package github.com/golangci/golangci-lint/loramac-node/src/boards/NucleoL152/cmsis/arm-std

Something weird is happening in this git submodule which is triggering broken go build (and other tools) behavior.

Cheers

[mvdan]

Closing per golangci/golangci-lint#3339 (comment).

[jdgordon]

@mvdan please reopen. After a bit more digging I'm far more confident this is a golang problem, not golangci-lint.

I did a bit more digging into the submodule that causes the problem, there are a number of deep directories which contain assembly files (.s). If they are removed then both tools go back to working correctly.

I have tried CGO_ENABLED=0 go build ./... but that isnt enough to stop it trying to walk the directories with assembly.

jonathan@fleetlaptop:~/git/testrepo$ CGO_ENABLED=1 go build ./...
pattern ./...: main module (gitlab.com/fleetspace/monoverse) does not contain package gitlab.com/fleetspace/monoverse/external/loramac-node/src/boards/NAMote72/cmsis/arm-std
jonathan@fleetlaptop:~/git/testrepo$ CGO_ENABLED=0 go build ./...
pattern ./...: main module (gitlab.com/fleetspace/monoverse) does not contain package gitlab.com/fleetspace/monoverse/external/loramac-node/src/boards/NAMote72/cmsis/arm-std
jonathan@fleetlaptop:~/git/testrepo$ find external/loramac-node/ -iname *.s -exec rm '{}' \;
jonathan@fleetlaptop:~/git/testrepo$ CGO_ENABLED=0 go build ./...
go: build output "myapp" already exists and is a directory

Please not the change in behaviour with and without those .s files.

[seankhliao]

Go code can contain assembly. Looks like this is working as intended.

[jdgordon]

These assembly files are NOT expected to be included in any go builds. They are apparently breaking the build because they are accessible by walking the entire filesystem tree under the go.mod file, but they are external files. It doesn't make sense that they should be able to break the build.

Further, the actual error is nonsensical. pattern ./...: main module (gitlab.com/fleetspace/monoverse) does not contain package gitlab.com/fleetspace/monoverse/external/loramac-node/src/boards/NAMote72/cmsis/arm-std This error gives the user absolutely no information on whats wrong.

Why is the go tools trying to bring in random .s files not associated with any .go files?

[bcmills]

Since there is no .go source file in the directory, it should not be treated as a Go package. The ./... pattern is supposed to omit non-package directories anyhow.

Moreover, if it were treated as a Go package, then the “does not contain package” error message would be misleading (because, as far as I can tell, that directory is in the main module).

[matloob]

Looking into this: here's a minimal reproducer script test:

go list ./...

-- go.mod --
module example.com/foo

go 1.20
-- bar/baz.s --
;/

Running it produces the following:

➜  z devgo test cmd/go -run TestScript/list_issue_56509
vcs-test.golang.org rerouted to http://127.0.0.1:59675
https://vcs-test.golang.org rerouted to https://127.0.0.1:59676
go test proxy running at GOPROXY=http://127.0.0.1:59677/mod
--- FAIL: TestScript (0.04s)
    --- FAIL: TestScript/list_issue_56509 (0.01s)
        script_test.go:134: 2022-11-02T16:27:36Z
        script_test.go:136: $WORK=/var/folders/qb/r47yxsb14sx6nz0m6_1pglm8004_sl/T/cmd-go-test-3123806881/tmpdir2199724403/list_issue_565092659442482
        script_test.go:154: 
            > go list ./...
            [stderr]
            pattern ./...: main module (example.com/foo) does not contain package example.com/foo/bar
        script_test.go:154: FAIL: list_issue_56509.txt:1: go list ./...: exit status 1
FAIL
FAIL	cmd/go	0.380s
FAIL

The interesting thing is that the contents of the .s file matter: if it's empty there's no error

[gopherbot]

Change https://go.dev/cl/447357 mentions this issue: go/build: don't add unparsable non-Go files to InvalidGoFiles

[jdgordon]

Thanks for reopening and looking into the bug :)