cmd/go: default to GO111MODULE=on

[rsc]

Back in Go 1.13 we tried defaulting to GO111MODULE=on but discovered a bunch of things that needed to be smoother before making it the default. Instead we added GO111MODULE=auto and made that the default, so that people with a go.mod could always get modules, but people without a go.mod would still get GOPATH mode.

The general plan since Go 1.11 to retire GOPATH mode has not changed. That's still an important step.

We believe Go 1.16 will have a smooth enough module experience to make it the default always. At that point GOPATH will be deprecated, to be removed entirely in some future Go version, perhaps Go 1.17.

We need to change the default soon, so that we can use it ourselves during the rest of the Go 1.16 cycle before shipping it in the release.

There are two important functionality changes remaining before we can change the default:

• Removing automatic updating of go.mod by most commands (cmd/go: default to & improve -mod=readonly #40728).
• A standalone (outside a module) command to download and install a binary, which will be go get pkg@version (go get pkg@latest if you want the latest version) (cmd/go: 'go install' should install executables in module mode outside a module #40276).

Once those are done, I believe we can flip the default. If there are any other blocking issues, please comment. Thank you.

[docmerlin]

I rather liked the gopath, it was a clean way to organize stuff, it made working on multiple inter-operating things much nicer, can we retain a way to keep it optionally... just have that not be the default?

[rsc]

I still check out my repos in "GOPATH organization" when working with modules, and you're welcome to do the same.

If you want changes in one module to be used by another, the way to do that in a go.mod is to add a replace line.
For example, if I check out golang.org/x/website and golang.org/x/tools to $GOPATH/src/golang.org/x/{website,tools}, then to make my local builds of website automatically use my changes in tools, I add this to website/go.mod:

replace golang.org/x/tools => ../tools

And when I'm done with local development I remove that line again. This keeps the intended dependencies explicit.

In contrast, the GOPATH approach of doing that replacement automatically means that your build is affected by old checkouts you happen to have lying around that you might have forgotten about. It means that the build you get on one machine can be different from another, even starting with the same version of the same top-level repo. And it means that the builds you get can be different from the ones another developer in the same project gets. Modules address all these reproducibility concerns.

In addition to reproducibility, modules provide a clear way to handle proxying and secure downloads. When you git clone a project and then grab its dependencies, those dependencies are being cryptographically checked to make sure they're the same bits the original developer used. The only trusted part is the top-level "git clone".

And for future evolution of Go itself, modules clearly mark which version of the Go language a particular tree of files is written in. This makes it possible to disable problematic features - for example, string(1), which many people think produces "1" but actually produces "\x01" (^A) - in later versions of Go by default while at the same time keeping older programs building (because they are explicitly marked as having been written for the older version of Go).

There's more I could list.

None of this is possible with GOPATH as it exists today. We can't move the ecosystem forward and start really depending on these important properties without retiring GOPATH.

(You might ask: why not just add those things to GOPATH? The answer is: we did, and the result is Go modules.)

[matthewmueller]

Thanks Go team for all your hard work on this!

Minor point: I noticed that go mod init is able to infer the module name when you're working inside GOPATH.

When you're outside of GOPATH, you get:

go: cannot determine module path for source directory /Users/m/Downloads (outside GOPATH, module path must be specified)

Example usage:
        'go mod init example.com/m' to initialize a v0 or v1 module
        'go mod init example.com/m/v2' to initialize a v2 module

Run 'go help mod init' for more information.

If GOPATH will be removed, could this feature be kept? Perhaps with a separate environment variable?

[rsc]

Yes, sorry for the confusion: GOPATH mode (GO111MODULE=off) will eventually be removed. The GOPATH environment variable will be sticking around, since it is used to derive the default install directory (GOPATH/bin) as well as the location of the module and build caches (GOPATH/pkg).

[mvdan]

The GOPATH environment variable will be sticking around, since it is used to derive the default install directory (GOPATH/bin) as well as the location of the module and build caches (GOPATH/pkg).

Thinking outloud, we could eventually deprecate GOPATH in favour of GOBIN, GOMODCACHE, and GOCACHE. We could keep support for it for a long time for the sake of backwards compatibility, but once modules are the only supported way to write Go code, it seems wrong to me to tell new Go developers to set up GOPATH.

[icholy]

@docmerlin I use this small git wrapper which clones into a GOPATH like layout https://github.com/icholy/git-get

[rsc]

@mvdan, I see very little benefit to deprecating the GOPATH environment variable. Today I can set one variable to say "put all the automatic Go stuff over here". Changing that to setting three different variables doesn't seem like a win at all, and what happens when we add a fourth thing?

[gopherbot]

Change https://golang.org/cl/255052 mentions this issue: cmd/go: default to GO111MODULE=on

[hyangah]

In addition to the functionality changes, I think we need facelifting of the cmd/go documentation to be more module oriented.

I just encountered an issue related to go run, mentioned here. The limitation is described in the go1.14 release note, but not in the command documentation yet. Personally I'd be happier if go run or go build can run outside modules. If they are no longer supported and are outside the modules roadmap, we need to make the differences and limitation clear in documentation.

I also noticed GOPATH is mentioned throughout the documentation. When defaulting to enable modules, they may cause confusion, especially for new users who are not aware of all the history.

[jayconrod]

@hyangah

About the documentation, thanks for bringing this up, I totally agree. I just opened #41427 to track updating the go help pages.

About go run and go build outside modules, #32027 was the discussion for the change in 1.14. If those commands should work differently, let's discuss that in a new issue soon. We're making a number of changes to the CLI this cycle like #40728 and #40276. Personally, I wouldn't mind something like #40276 for go run, i.e., go run example.com/cmd@v1.0.0 or go run example.com/cmd@latest with the same restrictions as for go install.

[gopherbot]

Change https://golang.org/cl/255398 mentions this issue: all: fix tests in preparation for GO111MODULE=on by default