Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: 'go list -m all' doesn't diagnose conflicts in non-imported packages #41652

Open
yihuang opened this issue Sep 27, 2020 · 4 comments
Open

Comments

@yihuang
Copy link

@yihuang yihuang commented Sep 27, 2020

Currently vgo2nix is using go list -mod mod -json -m all to find out all the dependencies of current project, but we find this command will output conflicts modules.

A simple example:

cat > go.mod << EOF
module testmod1

go 1.15

require (
	github.com/ugorji/go v1.1.4 // indirect
	github.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8 // indirect
)
EOF
cat > main.go << EOF
package main

import "fmt"

func main() {
	fmt.Println("hello world")
}
EOF

These too dependencies are in conflict, but not imported by code, so go build and go mod vendor runs just fine.
But go list -mod mod -m all will list them both.

Is it considered an issue of go list, or is there a command to serve our purpose?

@andybons
Copy link
Member

@andybons andybons commented Sep 29, 2020

@andybons andybons added this to the Unplanned milestone Sep 29, 2020
@bcmills
Copy link
Member

@bcmills bcmills commented Sep 29, 2020

go mod vendor copies in non-test dependencies matching the all package pattern. go list -m all reports the all module pattern.

The package namespace and module namespace are related, but not quite the same. The all module pattern includes (as you have seen) all of the dependencies imposed by the go.mod file, but the all package pattern (the one used by go build all and go mod vendor!) examines only those packages transitively imported by the main module. So there is no conflict in the dependencies of this module, because the only package that conflicts is not in the all package pattern.

The general solution is to give the go command the source code that you intend to build, so that go mod vendor can trace the import statements accurately. If that general solution does not suit your needs, we'll need to know more about the concrete problem you're trying to solve.

@bcmills bcmills changed the title cmd/go: `go list -mod mod -m all` don't return the same module list actual used when building or vendering cmd/go: 'go list -m all' doesn't diagnose conflicts in non-imported packages Sep 29, 2020
@yihuang
Copy link
Author

@yihuang yihuang commented Sep 29, 2020

The general solution is to give the go command the source code that you intend to build, so that go mod vendor can trace the import statements accurately. If that general solution does not suit your needs, we'll need to know more about the concrete problem you're trying to solve.

what vgo2nix do is try to get the locked dependency version informations from current source directory, and convert that information to nix expression and build in GOPATH mode, to get a more re-producible build (from nix perspective).
so i think maybe one possibility is to work on the all package list, and try to figure out the module list and versions?
another approach is just rely on go mod vendor, and treat the whole vendor directory as dependency, which is taken by buildGoModule, I wonder if the vgo2nix approach gonna work out?

@bcmills
Copy link
Member

@bcmills bcmills commented Sep 29, 2020

The all package list seems like a good starting point. Once we have lazy loading (#36460), you may be able to do something simpler using the original go.mod file and replace directives.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.