Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: recurses into directories only containing assembly .s files #56509

Closed
jdgordon opened this issue Nov 1, 2022 · 8 comments
Closed

cmd/go: recurses into directories only containing assembly .s files #56509

jdgordon opened this issue Nov 1, 2022 · 8 comments
Labels
FrozenDueToAge GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Milestone

Comments

@jdgordon
Copy link

jdgordon commented Nov 1, 2022

What version of Go are you using (go version)?

$ go version
go version go1.19.1 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/jonathan/.cache/go-build"
GOENV="/home/jonathan/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/jonathan/go/pkg/mod"
GONOPROXY="gitlab.com/fleetspace"
GONOSUMDB="gitlab.com/fleetspace"
GOOS="linux"
GOPATH="/home/jonathan/go"
GOPRIVATE="gitlab.com/fleetspace"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/jonathan/go/go1.19.1"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/jonathan/go/go1.19.1/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.19.1"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/jonathan/git/golangci-lint/go.mod"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build872883275=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I originally filed this issue with golangci-lint however they are pushing back that this is a go tooling problem. golangci/golangci-lint#3339

The Issue is that after adding a git submodule of a completely unrelated project into any part of the tree under the go.mod, attempting to build gives the wonderful error as follows

 go build ./...
pattern ./...: main module (github.com/golangci/golangci-lint) does not contain package github.com/golangci/golangci-lint/loramac-node/src/boards/NucleoL152/cmsis/arm-std

I have reproduced this with a clean checkout of the golangci-lint repo, as well as creating a test repo with nothing but the git submodule. Both show the same behavior.

What did you expect to see?

no problems with go build or golangci-lint when running with random unrelated git tree in the workdir.

What did you see instead?

This is a copy of my terminal showing how to reproduce

jonathan@fleetlaptop:~$ cd git/
jonathan@fleetlaptop:~/git$ git clone https://github.com/golangci/golangci-lint.git
Cloning into 'golangci-lint'...
remote: Enumerating objects: 22754, done.
remote: Counting objects: 100% (24/24), done.
remote: Compressing objects: 100% (16/16), done.
remote: Total 22754 (delta 14), reused 17 (delta 8), pack-reused 22730
Receiving objects: 100% (22754/22754), 39.55 MiB | 5.99 MiB/s, done.
Resolving deltas: 100% (13907/13907), done.
jonathan@fleetlaptop:~/git$ cd golangci-lint/
jonathan@fleetlaptop:~/git/golangci-lint$ ~/go/go1.17.13/bin/go
go             gofmt          golangci-lint  
jonathan@fleetlaptop:~/git/golangci-lint$ ~/go/go1.17.13/bin/golangci-lint run
jonathan@fleetlaptop:~/git/golangci-lint$ git submodule add https://github.com/lora-net/loramac-node.git
Cloning into '/home/jonathan/git/golangci-lint/loramac-node'...
remote: Enumerating objects: 28497, done.
remote: Counting objects: 100% (321/321), done.
remote: Compressing objects: 100% (144/144), done.
remote: Total 28497 (delta 190), reused 266 (delta 171), pack-reused 28176
Receiving objects: 100% (28497/28497), 27.37 MiB | 6.29 MiB/s, done.
Resolving deltas: 100% (18361/18361), done.
jonathan@fleetlaptop:~/git/golangci-lint$ ~/go/go1.17.13/bin/golangci-lint run
ERRO [linters_context] typechecking error: pattern ./...: main module (github.com/golangci/golangci-lint) does not contain package github.com/golangci/golangci-lint/loramac-node/src/boards/NAMote72/cmsis/arm-std 
jonathan@fleetlaptop:~/git/golangci-lint$ git diff
jonathan@fleetlaptop:~/git/golangci-lint$ git diff --staged
diff --git a/.gitmodules b/.gitmodules
new file mode 100644
index 00000000..0aad7827
--- /dev/null
+++ b/.gitmodules
@@ -0,0 +1,3 @@
+[submodule "loramac-node"]
+       path = loramac-node
+       url = https://github.com/lora-net/loramac-node.git
diff --git a/loramac-node b/loramac-node
new file mode 160000
index 00000000..7f25f7ad
--- /dev/null
+++ b/loramac-node
@@ -0,0 +1 @@
+Subproject commit 7f25f7adb5b9bf7142f98f77c68363e8b1f7e8ee
jonathan@fleetlaptop:~/git/golangci-lint$ go build ./...
pattern ./...: main module (github.com/golangci/golangci-lint) does not contain package github.com/golangci/golangci-lint/loramac-node/src/boards/NucleoL152/cmsis/arm-std

Something weird is happening in this git submodule which is triggering broken go build (and other tools) behavior.

Cheers

@mvdan
Copy link
Member

mvdan commented Nov 1, 2022

Closing per golangci/golangci-lint#3339 (comment).

@mvdan mvdan closed this as not planned Won't fix, can't repro, duplicate, stale Nov 1, 2022
@jdgordon
Copy link
Author

jdgordon commented Nov 2, 2022

@mvdan please reopen. After a bit more digging I'm far more confident this is a golang problem, not golangci-lint.

I did a bit more digging into the submodule that causes the problem, there are a number of deep directories which contain assembly files (.s). If they are removed then both tools go back to working correctly.

I have tried CGO_ENABLED=0 go build ./... but that isnt enough to stop it trying to walk the directories with assembly.

jonathan@fleetlaptop:~/git/testrepo$ CGO_ENABLED=1 go build ./...
pattern ./...: main module (gitlab.com/fleetspace/monoverse) does not contain package gitlab.com/fleetspace/monoverse/external/loramac-node/src/boards/NAMote72/cmsis/arm-std
jonathan@fleetlaptop:~/git/testrepo$ CGO_ENABLED=0 go build ./...
pattern ./...: main module (gitlab.com/fleetspace/monoverse) does not contain package gitlab.com/fleetspace/monoverse/external/loramac-node/src/boards/NAMote72/cmsis/arm-std
jonathan@fleetlaptop:~/git/testrepo$ find external/loramac-node/ -iname *.s -exec rm '{}' \;
jonathan@fleetlaptop:~/git/testrepo$ CGO_ENABLED=0 go build ./...
go: build output "myapp" already exists and is a directory

Please not the change in behaviour with and without those .s files.

@seankhliao
Copy link
Member

Go code can contain assembly. Looks like this is working as intended.

@jdgordon
Copy link
Author

jdgordon commented Nov 2, 2022

These assembly files are NOT expected to be included in any go builds. They are apparently breaking the build because they are accessible by walking the entire filesystem tree under the go.mod file, but they are external files. It doesn't make sense that they should be able to break the build.

Further, the actual error is nonsensical. pattern ./...: main module (gitlab.com/fleetspace/monoverse) does not contain package gitlab.com/fleetspace/monoverse/external/loramac-node/src/boards/NAMote72/cmsis/arm-std This error gives the user absolutely no information on whats wrong.

Why is the go tools trying to bring in random .s files not associated with any .go files?

@bcmills bcmills reopened this Nov 2, 2022
@bcmills
Copy link
Member

bcmills commented Nov 2, 2022

Since there is no .go source file in the directory, it should not be treated as a Go package. The ./... pattern is supposed to omit non-package directories anyhow.

Moreover, if it were treated as a Go package, then the “does not contain package” error message would be misleading (because, as far as I can tell, that directory is in the main module).

@bcmills bcmills added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. GoCommand cmd/go labels Nov 2, 2022
@bcmills bcmills added this to the Backlog milestone Nov 2, 2022
@matloob
Copy link
Contributor

matloob commented Nov 2, 2022

Looking into this: here's a minimal reproducer script test:

go list ./...

-- go.mod --
module example.com/foo

go 1.20
-- bar/baz.s --
;/

Running it produces the following:

➜  z devgo test cmd/go -run TestScript/list_issue_56509
vcs-test.golang.org rerouted to http://127.0.0.1:59675
https://vcs-test.golang.org rerouted to https://127.0.0.1:59676
go test proxy running at GOPROXY=http://127.0.0.1:59677/mod
--- FAIL: TestScript (0.04s)
    --- FAIL: TestScript/list_issue_56509 (0.01s)
        script_test.go:134: 2022-11-02T16:27:36Z
        script_test.go:136: $WORK=/var/folders/qb/r47yxsb14sx6nz0m6_1pglm8004_sl/T/cmd-go-test-3123806881/tmpdir2199724403/list_issue_565092659442482
        script_test.go:154: 
            > go list ./...
            [stderr]
            pattern ./...: main module (example.com/foo) does not contain package example.com/foo/bar
        script_test.go:154: FAIL: list_issue_56509.txt:1: go list ./...: exit status 1
FAIL
FAIL	cmd/go	0.380s
FAIL

The interesting thing is that the contents of the .s file matter: if it's empty there's no error

@seankhliao seankhliao changed the title go build: attempts to recurse into non-golang directories breaks the build cmd/go: recurses into directories only containing assembly .s files Nov 2, 2022
@gopherbot
Copy link

Change https://go.dev/cl/447357 mentions this issue: go/build: don't add unparsable non-Go files to InvalidGoFiles

@jdgordon
Copy link
Author

jdgordon commented Nov 2, 2022

Thanks for reopening and looking into the bug :)

@golang golang locked and limited conversation to collaborators Nov 3, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge GoCommand cmd/go NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Projects
None yet
Development

No branches or pull requests

6 participants