Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: resolve local paths to modules in subdirectory replacements #43733

Open
jayconrod opened this issue Jan 15, 2021 · 8 comments
Open

cmd/go: resolve local paths to modules in subdirectory replacements #43733

jayconrod opened this issue Jan 15, 2021 · 8 comments
Labels
GoCommand modules NeedsInvestigation
Milestone

Comments

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Jan 15, 2021

What version of Go are you using (go version)?

$ go version
go version go1.16beta1 darwin/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/jayconrod/Library/Caches/go-build"
GOENV="/Users/jayconrod/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/jayconrod/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/jayconrod/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/opt/go/installed"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/opt/go/installed/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.16beta1"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/jayconrod/Code/test3/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/rq/x0692kqj6ml8cvrhcqh5bswc008xj1/T/go-build3849412287=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Run go list or any other module-aware command on an absolute or relative path (starting with ./) that points to a subdirectory inside a separate module which provides a replacement.

go list ./sub
go list $(pwd)/sub

-- go.mod --
module example.com/m

go 1.16

require example.com/m/sub v0.0.0

replace example.com/m/sub v0.0.0 => ./sub
-- sub/go.mod --
module example.com/m/sub

go 1.16
-- sub/sub.go --
package sub

What did you expect to see?

go list should succeed and print the package path. In the example above, it should print example.com/m/sub.

What did you see instead?

go list fails with:

main module (example.com/m) does not contain package example.com/m/sub

It looks like the go command resolves the path argument to a directory within the module before checking whether it might be part of a replacement module in another directory.

Other notes

This issue is related to the thread go list across multiple modules on golang-nuts.

@jayconrod jayconrod added the NeedsFix label Jan 15, 2021
@jayconrod jayconrod added this to the Go1.17 milestone Jan 15, 2021
@jayconrod jayconrod added the okay-after-beta1 label May 17, 2021
@gopherbot
Copy link

@gopherbot gopherbot commented Jul 17, 2021

Change https://golang.org/cl/335269 mentions this issue: cmd/go: expand local directory in replace directive

@bcmills bcmills removed this from the Go1.17 milestone Jul 22, 2021
@bcmills bcmills added this to the Go1.18 milestone Jul 22, 2021
@bcmills bcmills removed the okay-after-beta1 label Jul 22, 2021
@hyangah
Copy link
Contributor

@hyangah hyangah commented Jul 22, 2021

Is this proposal to cover only the case where the module in the subdirectory is mentioned in the main module? Or is it going to allow to run go commands with subdirectories whose module aren't listed in the main module? For example, the current x/tools repo has golang.org/x/tools/gopls module in the gopls directory and is not a dependency of golang.org/x/tools. Will this change allow go list ./gopls from the the repo root to work?

@jayconrod
Copy link
Contributor Author

@jayconrod jayconrod commented Jul 22, 2021

Will this change allow go list ./gopls from the the repo root to work?

No, that shouldn't work, though an error message probably ought to say why. After this is fixed, go list should (still) only be able to load packages from modules in the build list.

@iwdgo
Copy link
Contributor

@iwdgo iwdgo commented Nov 13, 2021

On a recent tip, go list seems to behave as expected:
The directory ...\43733 has the structure described in the issue.
The sub-directory sub is a sub-module in the context of the main module and a package in a context outside the module. Replacing the required sub-module by a local package looks like a mistake from the module perspective.

GOPATH\Issues\43733>gotip version
go version devel go1.18-95d0657670 Fri Nov 12 16:58:34 2021 +0000 windows/amd64

GOPATH\Issues\43733>set GO111MODULE
Environment variable GO111MODULE not defined

GOPATH\Issues\43733>gotip list
no Go files in GOPATH\Issues\43733

GOPATH\Google\Issues\43733>gotip list -m
example.com/m

example.com/m is a module as a go.mod is present in its main directory.

GOPATH\Google\Issues\43733>gotip list -m ...
example.com/m
example.com/m/sub v0.0.0 => ./sub

example.com/m/sub is a sub-module in this context. It is replaced as a package in the local sub-directory.

GOPATH\Google\Issues\43733>gotip list -m ./sub
go: cannot use relative path ./sub to specify module

GOPATH\Google\Issues\43733>gotip list ./sub
main module (example.com/m) does not contain package example.com/m/sub

Errors seem correct as the sub directory is a package in this context and cannot be listed as a module.
When listing packages,, the main module does not contain the package example.com/m/sub but a module as go.mod is present. So there is an error. The main module requires a package which is in fact a sub-module.

GOPATH\Google\Issues\43733>gotip list ...
archive/tar
(all golang packages)
example.com/m/sub

The included directory is seen as a package.

No change in results when go.mod is set to 1.16 or 1.17.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Nov 17, 2021

This is in the Go1.18 milestone. Is it likely to happen for 1.18? Thanks.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Jan 28, 2022

@bcmills This is in the 1.18 milestone; time to move to 1.19? Thanks.

@bcmills bcmills removed this from the Go1.18 milestone Feb 1, 2022
@bcmills bcmills added this to the Backlog milestone Feb 1, 2022
@bcmills
Copy link
Member

@bcmills bcmills commented Feb 1, 2022

As of Go 1.18 this is also relevant to paths that could refer to packages in other modules in the same workspace.

The interaction between this and ... patterns could be a bit confusing, though. In the original example, ./sub would resolve to a package in a different module, but would ./... enumerate that package? (It doesn't today.)

@bcmills bcmills added the NeedsInvestigation label Feb 1, 2022
@gopherbot gopherbot removed the NeedsFix label Feb 1, 2022
@bcmills bcmills added the GoCommand label Feb 1, 2022
@iwdgo
Copy link
Contributor

@iwdgo iwdgo commented Feb 20, 2022

Using go1.18rc1 and setting go.mod version to 1.18, the behavior of the test of the issue evolves but seems in line with expectations.

The warning below should be considered as correct as the sub-directory is a module (go.mod is present) and a not a package.

>go list ./...
go: warning: "./..." matched no packages

When expecting all of them:

>go list ...
archive/tar
... (abridged as all packages of go source are listed)
example.com/m/sub

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
GoCommand modules NeedsInvestigation
Projects
None yet
Development

No branches or pull requests

6 participants