Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: default to GO111MODULE=on #41330

Closed
rsc opened this issue Sep 10, 2020 · 13 comments
Closed

cmd/go: default to GO111MODULE=on #41330

rsc opened this issue Sep 10, 2020 · 13 comments
Labels
Milestone

Comments

@rsc
Copy link
Contributor

@rsc rsc commented Sep 10, 2020

Back in Go 1.13 we tried defaulting to GO111MODULE=on but discovered a bunch of things that needed to be smoother before making it the default. Instead we added GO111MODULE=auto and made that the default, so that people with a go.mod could always get modules, but people without a go.mod would still get GOPATH mode.

The general plan since Go 1.11 to retire GOPATH mode has not changed. That's still an important step.

We believe Go 1.16 will have a smooth enough module experience to make it the default always. At that point GOPATH will be deprecated, to be removed entirely in some future Go version, perhaps Go 1.17.

We need to change the default soon, so that we can use it ourselves during the rest of the Go 1.16 cycle before shipping it in the release.

There are two important functionality changes remaining before we can change the default:

  • Removing automatic updating of go.mod by most commands (#40728).
  • A standalone (outside a module) command to download and install a binary, which will be go get pkg@version (go get pkg@latest if you want the latest version) (#40276).

Once those are done, I believe we can flip the default. If there are any other blocking issues, please comment. Thank you.

@rsc rsc added this to the Go1.16 milestone Sep 10, 2020
@docmerlin
Copy link

@docmerlin docmerlin commented Sep 10, 2020

I rather liked the gopath, it was a clean way to organize stuff, it made working on multiple inter-operating things much nicer, can we retain a way to keep it optionally... just have that not be the default?

@rsc
Copy link
Contributor Author

@rsc rsc commented Sep 10, 2020

I still check out my repos in "GOPATH organization" when working with modules, and you're welcome to do the same.

If you want changes in one module to be used by another, the way to do that in a go.mod is to add a replace line.
For example, if I check out golang.org/x/website and golang.org/x/tools to $GOPATH/src/golang.org/x/{website,tools}, then to make my local builds of website automatically use my changes in tools, I add this to website/go.mod:

replace golang.org/x/tools => ../tools

And when I'm done with local development I remove that line again. This keeps the intended dependencies explicit.

In contrast, the GOPATH approach of doing that replacement automatically means that your build is affected by old checkouts you happen to have lying around that you might have forgotten about. It means that the build you get on one machine can be different from another, even starting with the same version of the same top-level repo. And it means that the builds you get can be different from the ones another developer in the same project gets. Modules address all these reproducibility concerns.

In addition to reproducibility, modules provide a clear way to handle proxying and secure downloads. When you git clone a project and then grab its dependencies, those dependencies are being cryptographically checked to make sure they're the same bits the original developer used. The only trusted part is the top-level "git clone".

And for future evolution of Go itself, modules clearly mark which version of the Go language a particular tree of files is written in. This makes it possible to disable problematic features - for example, string(1), which many people think produces "1" but actually produces "\x01" (^A) - in later versions of Go by default while at the same time keeping older programs building (because they are explicitly marked as having been written for the older version of Go).

There's more I could list.

None of this is possible with GOPATH as it exists today. We can't move the ecosystem forward and start really depending on these important properties without retiring GOPATH.

(You might ask: why not just add those things to GOPATH? The answer is: we did, and the result is Go modules.)

@matthewmueller
Copy link

@matthewmueller matthewmueller commented Sep 10, 2020

Thanks Go team for all your hard work on this!

Minor point: I noticed that go mod init is able to infer the module name when you're working inside GOPATH.

When you're outside of GOPATH, you get:

go: cannot determine module path for source directory /Users/m/Downloads (outside GOPATH, module path must be specified)

Example usage:
        'go mod init example.com/m' to initialize a v0 or v1 module
        'go mod init example.com/m/v2' to initialize a v2 module

Run 'go help mod init' for more information.

If GOPATH will be removed, could this feature be kept? Perhaps with a separate environment variable?

@rsc
Copy link
Contributor Author

@rsc rsc commented Sep 11, 2020

Yes, sorry for the confusion: GOPATH mode (GO111MODULE=off) will eventually be removed. The GOPATH environment variable will be sticking around, since it is used to derive the default install directory (GOPATH/bin) as well as the location of the module and build caches (GOPATH/pkg).

@ALTree ALTree added the NeedsFix label Sep 12, 2020
@mvdan
Copy link
Member

@mvdan mvdan commented Sep 12, 2020

The GOPATH environment variable will be sticking around, since it is used to derive the default install directory (GOPATH/bin) as well as the location of the module and build caches (GOPATH/pkg).

Thinking outloud, we could eventually deprecate GOPATH in favour of GOBIN, GOMODCACHE, and GOCACHE. We could keep support for it for a long time for the sake of backwards compatibility, but once modules are the only supported way to write Go code, it seems wrong to me to tell new Go developers to set up GOPATH.

@icholy
Copy link

@icholy icholy commented Sep 13, 2020

@docmerlin I use this small git wrapper which clones into a GOPATH like layout https://github.com/icholy/git-get

@rsc
Copy link
Contributor Author

@rsc rsc commented Sep 15, 2020

@mvdan, I see very little benefit to deprecating the GOPATH environment variable. Today I can set one variable to say "put all the automatic Go stuff over here". Changing that to setting three different variables doesn't seem like a win at all, and what happens when we add a fourth thing?

@gopherbot
Copy link

@gopherbot gopherbot commented Sep 15, 2020

Change https://golang.org/cl/255052 mentions this issue: cmd/go: default to GO111MODULE=on

@hyangah
Copy link
Contributor

@hyangah hyangah commented Sep 16, 2020

In addition to the functionality changes, I think we need facelifting of the cmd/go documentation to be more module oriented.

I just encountered an issue related to go run, mentioned here. The limitation is described in the go1.14 release note, but not in the command documentation yet. Personally I'd be happier if go run or go build can run outside modules. If they are no longer supported and are outside the modules roadmap, we need to make the differences and limitation clear in documentation.

I also noticed GOPATH is mentioned throughout the documentation. When defaulting to enable modules, they may cause confusion, especially for new users who are not aware of all the history.

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Sep 16, 2020

@hyangah

About the documentation, thanks for bringing this up, I totally agree. I just opened #41427 to track updating the go help pages.

About go run and go build outside modules, #32027 was the discussion for the change in 1.14. If those commands should work differently, let's discuss that in a new issue soon. We're making a number of changes to the CLI this cycle like #40728 and #40276. Personally, I wouldn't mind something like #40276 for go run, i.e., go run example.com/cmd@v1.0.0 or go run example.com/cmd@latest with the same restrictions as for go install.

@gopherbot
Copy link

@gopherbot gopherbot commented Sep 16, 2020

Change https://golang.org/cl/255398 mentions this issue: all: fix tests in preparation for GO111MODULE=on by default

@nightlyone
Copy link
Contributor

@nightlyone nightlyone commented Sep 17, 2020

We would now need new defaults for GOMODCACHE and GOBIN before doing the "deprecating GOPATH" part of this proposal.

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Sep 17, 2020

@nightlyone We're not planning to deprecate the GOPATH environment variable. See the discussion at this comment above.

gopherbot pushed a commit to golang/tools that referenced this issue Sep 17, 2020
This CL does not fix failures in ./gopls/internal/regtest, which will
be fixed separately.

In refactor/rename.TestDiff, add a go.mod file.

In internal/imports.ProcessEnv.buildContext, set an I/O hook if
GO111MODULE=off in ProcessEnv but not in the current process's
environment.

Context allows the user to set GOPATH, GOOS, GOARCH, and a few other
environment variables, but not GO111MODULE. Context.Import may return
different results than packages.Load if the latter is invoked with a
GO111MODULE value that differs from the caller's environment. Setting
an I/O hook forces Import to run in GOPATH mode, not invoking 'go list'.
This is undocumented, but it should be stable while GOPATH is
supported.

For golang/go#41330

Change-Id: I5679e8941e32dc95b05c234cb2e3fec5cabebced
Reviewed-on: https://go-review.googlesource.com/c/tools/+/255398
Run-TryBot: Jay Conrod <jayconrod@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Jay Conrod <jayconrod@google.com>
@gopherbot gopherbot closed this in b4ea672 Sep 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
10 participants
You can’t perform that action at this time.