Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: cmd/go: maintain 'classic' vendor behaviour #44519

Open
kungfusheep opened this issue Feb 23, 2021 · 3 comments
Open

proposal: cmd/go: maintain 'classic' vendor behaviour #44519

kungfusheep opened this issue Feb 23, 2021 · 3 comments
Labels
Projects
Milestone

Comments

@kungfusheep
Copy link

@kungfusheep kungfusheep commented Feb 23, 2021

Proposal: Maintain 'classic' vendor behaviour

This is a proposal to keep resolution of 'vendored' code consistent with the behaviour possible in go1.16 < versions of the compiler.

The express of intent to remove GOPATH from the compiler toolchain as soon as go1.17, and furthermore the GO111MODULE flag, combined with the fact -mod=vendor ignores vendor directories in locations other than the main module's root directory; means applications designed and written in the years before go mod could no longer compile.

To be clear, this proposal does not make an argument to keep GOPATH code resolution 1:1, but to allow code to be resolved from both the project root's vendor directory and any vendor directories found to be relative to import statements within.

In order to take any subsequent security updates and language features, teams are faced with non-insignificant engineering and logistical tasks to make this possible. For us and I imagine many others this is multiplied across a number of applications into the hundreds.

Our specific set up is to use git submodules of packages into the vendor directory, which is sufficient for our use-case of a large body of internally owned packages.

The applications have grown and adapted these structures over time based on the features available in the compiler until now, and the fact that until recently there was no 1st party solution for package management.

Removing the ability to resolve code in this way feels inconsistent with the spirit of the below paragraph taken from the go1compat document, which I appreciate people are likely familiar with.

It is intended that programs written to the Go 1 specification will continue to compile and run correctly, unchanged, over the lifetime of that specification. At some indefinite point, a Go 2 specification may arise, but until that time, Go programs that work today should continue to work even as future "point" releases of Go 1 arise (Go 1.1, Go 1.2, etc.).

Example

❯ go1.16 build
go: cannot find main module, but found .git/config in /Users/username/code/go/src/myapp
        to create a module there, run:
        go mod init

❯ go mod init
go: creating new go.mod: module myapp

❯ go1.16 build
vendor/git.lb.local/pkg/example/example.go:17:2: cannot find package "." in:
        /Users/username/code/go/src/myapp/vendor/github.com/protobuf/proto

In the above example, github.com/protobuf sits in the folder myapp/vendor/git.lb.local/pkg/example/vendor but is unable to be resolved. In go 1.16 this can be mitigated via the GO111MODULE flag.

References

@gopherbot gopherbot added this to the Proposal milestone Feb 23, 2021
@gopherbot gopherbot added the Proposal label Feb 23, 2021
@ianlancetaylor ianlancetaylor changed the title Proposal: Maintain 'classic' vendor behaviour proposal: cmd/go: maintain 'classic' vendor behaviour Feb 23, 2021
@ianlancetaylor ianlancetaylor added this to Incoming in Proposals Feb 23, 2021
@tv42
Copy link

@tv42 tv42 commented Feb 24, 2021

To continue using your git submodule layout while using modules, I think you can add a replace line to go.mod for each of the modules that reside in there, with a relative path starting with ./. Might be less confusing to combine that with a move from vendor/ to third_party/.

@rsc
Copy link
Contributor

@rsc rsc commented Feb 24, 2021

Unfortunately, "classic" vendor lets you end up with multiple copies of different (or even identical) minor versions of a package in the same build, which causes all manner of headaches. Adding that to module mode would introduce a special case that violates one of module mode's basic guarantees.

The applications have grown and adapted these structures over time based on the features available in the compiler until now, and the fact that until recently there was no 1st party solution for package management.

I apologize for the work you'll need to do to deduplicate your vendor directories into a single top-level vendor directory. All I can say is that if you make each one use a go.mod instead of vendor, that will set you up much better for understanding which code versions you are actually using, for updating in response to security fixes, and so on. It really does lead to a better, more agile setup.

I would push back a little on the "until recently" part. Go modules were first made available for use in Go 1.11 (August 2018), and we announced our intent to remove GOPATH in the "Go Modules in 2019" blog post (December 2018). Also, Go 1.16 will be supported until the release of Go 1.18 (~February 2022).

Removing the ability to resolve code in this way feels inconsistent with the spirit of the below paragraph taken from the go1compat document, which I appreciate people are likely familiar with.

Note also that the document explicitly excludes tools such as the Go command. It is concerned with the Go language and the standard library.

We do try to follow the spirit of the document generally, and again we are aware that it is painful to update old code trees. We don't intend to make a break like this again any time soon, if ever. But this one really is necessary for the progress of the Go ecosystem.

@ssxcxS45XT
Copy link

@ssxcxS45XT ssxcxS45XT commented Feb 24, 2021

@rsc You statement assumes that people always uses other dependencies or such dependences always have version problems. But these simply are not always true. When those cases don't occur, it is unnecesary to enforce all Go code must be in a module. For this reason, the classical behavior should be maintained.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Proposals
Incoming
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants