Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: go mod download does not work with submodules #31176

Closed
marwan-at-work opened this issue Apr 1, 2019 · 6 comments

Comments

Projects
None yet
2 participants
@marwan-at-work
Copy link
Contributor

commented Apr 1, 2019

Summary

https://github.com/NYTimes/gizmo has two go.mod files: one at the root level and one inside the examples folder.

If I try to go mod download the top level module as such:

GO111MODULE=on go mod download -json github.com/NYTimes/gizmo/@v/v1.2.4

Everything works correctly, but the .zip file that gets created does not have the examples folder...I imagine this is expected behavior because the examples folder and all of its sub-packages have their own go.mod file independent of the root one.

Therefore, I would expect that go mod download github.com/NYTimes/gizmo/examples/@v/v1.2.4 to work properly but currently it's not working (see error below)

From the above behavior, I have two thoughts:

  1. It would be nice if we can somehow conclude that a repository has multiple modules. This behavior comes in handy when it comes to code discovery and documentation websites such as when godoc.org supports modules. For example, godoc.org already has a Subdirectories section so it's become an expected behavior to see all subdirectories so we should at least see the examples folder or be hinted at a sub-module so we wouldn't be surprised if that folder is missing. I have a few ideas as to how we can find out about them, but first I'd like to see if that idea is well-accepted in the first place.

  2. We shouldn't error out on go mod downloading a sub-module especially that it's not included in the root-level module zip file.

What version of Go are you using (go version)?

$ go version
go version go1.12 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
A8495:~ 208581$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/208581/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/208581/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/n4/35bgdnkd6vlgqrdzbyl0x1ycmhcndx/T/go-build121030453=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

go mod download -json github.com/NYTimes/gizmo/examples@v1.2.4

What did you expect to see?

The sub-module correctly downloaded to $GOPATH/pkg/mod

What did you see instead?

go: finding github.com/NYTimes/gizmo/examples v1.2.4
{
	"Path": "github.com/NYTimes/gizmo/examples",
	"Version": "v1.2.4",
	"Error": "unknown revision examples/v1.2.4"
}
@bcmills

This comment has been minimized.

Copy link
Member

commented Apr 1, 2019

The v1.2.4 tag applies only to the module at the repo root. A module within the repo at the relative path examples/ would need a corresponding tag examples/v1.2.4.

@bcmills

This comment has been minimized.

Copy link
Member

commented Apr 1, 2019

I think that makes this a duplicate of #30850, but please reopen if you disagree.

@bcmills bcmills closed this Apr 1, 2019

@marwan-at-work

This comment has been minimized.

Copy link
Contributor Author

commented Apr 1, 2019

@bcmills that solves issue no.2 but not issue no.1 -- For issue 1: are there any conclusions on whether go mod downlaod (or any go sub-command) should be able to tell us if a module has sub-modules? Purely for discovery/documentation reasons.

Thanks :)

@bcmills

This comment has been minimized.

Copy link
Member

commented Apr 1, 2019

The module proxy protocol doesn't include nested modules within the parent module, so in general there is no way for the go command to indicate that they exist without actively searching for them, and we don't want the behavior to differ between proxied and non-proxied go commands without a very compelling reason.

(We certainly do want to support module discovery and documentation, but go mod download is not a very good fit for that use-case. Stay tuned!)

@bcmills

This comment has been minimized.

Copy link
Member

commented May 28, 2019

We certainly do want to support module discovery and documentation, but go mod download is not a very good fit for that use-case. Stay tuned!

https://index.golang.org/ now serves a feed of known module paths and versions at https://index.golang.org/index. That should be suitable for discovering modules programmatically — please file issues if you find that it is not.

@marwan-at-work

This comment has been minimized.

Copy link
Contributor Author

commented May 28, 2019

@bcmills I imagine for say github.com/user/pkg we can ping index.golang.org/index to see if any of those lines include a github.com/user/pkg/ prefix, however there are two drawbacks:

  1. If the module has not been discovered by index.golang.org yet, we're not gonna find submodules.
  2. If the module is private and not public, there's no way to use index.golang.org.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.