cmd/go: deprecate installing binaries using 'go get' in Go 1.17 and make 'go get -d' the default behavior

[bcmills]

Proposal #40276 added a versioned go install variant that works outside of a module.

As part of the changes approved in that proposal, we plan to deprecate the use of the go get to install binaries, and make go get -d (which downloads source code for but does not build the requested packages) the default behavior.

We had planned to warn about the use of go get to install binaries in Go 1.16 and make it the default in Go 1.17. However, in #42885 we decided to also delay the warning until Go 1.17, so that third-party projects that support the most recent two major Go releases (as the Go project itself does) can give users a single non-deprecated install command, rather than a confusing menu of commands that vary by Go version.

I don't see an issue filed yet to track that change in Go 1.17, so this is that tracking issue. (CC @jayconrod @matloob)

[bcmills]

It is not clear to me whether we should print a deprecation notice in Go 1.17 and change the default behavior in Go 1.18, or change the default behavior in 1.17 and merely print a notice informing the user of the change if their go get pattern happens to match a package main.

[andig]

As part of the changes approved in that proposal, we plan to deprecate the use of the go get to install binaries, and make go get -d (which downloads source code for but does not build the requested packages) the default behavior.

I‘m all for this. Its a regular source of confusion that go get fails when I pull the sources for later cross-compile and the go get fails with architecture-specific error messages.

[gopherbot]

This issue is currently labeled as early-in-cycle for Go 1.17.
That time is now, so a friendly reminder to look at it again.

[ohir]

change the default behavior in 1.17 and merely print a notice informing the user of the change if their go get pattern happens to match a package main.

The sooner the better. Just add a short instruction for the non-Go user stumbling upon older source that "Since Go 1.17 to install an app from the source you need to use go install instead of go get". I'd also suggest adding the -u flag to the install - silent or with a compact message that "-u is not needed with install".

[dmitshur]

Friendly ping @bcmills, @jayconrod, @matloob since this is a release blocking issue and has an early in cycle label.

[gopherbot]

Change https://golang.org/cl/305670 mentions this issue: cmd/go: print deprecation notice for 'go get cmd'

[gopherbot]

Change https://golang.org/cl/305649 mentions this issue: _content/doc: add a page explaining 'go get' deprecation

[fzipp]

If installing binaries using 'go get' is deprecated, the Go documentation should lead by example and use the recommended 'go install' command. There are:

• contribute.html
  go get -u golang.org/x/review/git-codereview
  go get -u golang.org/x/tools/cmd/go-contrib-init
  go get -u golang.org/x/review/git-codereview

• docs.html
  go get golang.org/x/tour

• install-source.html
  go get golang.org/x/tools/cmd/godoc (x2)

• manage-install.html
  go get golang.org/dl/go1.10.7

Also, many README files in the Go repositories instruct the user to install binaries using 'go get':

https://github.com/golang/blog
https://github.com/golang/dl
https://github.com/golang/example
https://github.com/golang/lint
https://github.com/golang/perf
https://github.com/golang/review
https://github.com/golang/tools
https://github.com/golang/tour
https://github.com/golang/appengine

[toothrot]

@bcmills We're getting close to the freeze. Just checking in as this issue is both early-in-cycle and a release-blocker. Any updates?

[bcmills]

Jay implemented the deprecation notice in CL 305670. I believe we're planning to actually change the default behavior in 1.18, so I think we're done with the 1.17 parts of this.)

(I'd like to confirm that understanding with Jay after he gets back, though. Leaving as release-blocker and marking okay-after-beta1 to make sure we do that.)

[jayconrod]

@bcmills comment is correct.

I still need to add a release note and update documentation for 1.17. After that, we can re-milestone to 1.18 for the default behavior change.

[gopherbot]

Change https://golang.org/cl/318570 mentions this issue: cmd/go: don't print a 'go get' deprecation notices in the main module

[pellared]

Is not the go get approach safer? Does it not validate the modules against go.sum? For sure it is bypassed when using go install.

If what I suspect is true then usage of go install is more vulnerable to supply chain attacks.

[gopherbot]

Change https://golang.org/cl/349997 mentions this issue: cmd/go: 'go get' no longer builds or installs packages

[jayconrod]

It is not clear to me whether we should print a deprecation notice in Go 1.17 and change the default behavior in Go 1.18, or change the default behavior in 1.17 and merely print a notice informing the user of the change if their go get pattern happens to match a package main.

In Go 1.17, go get without -d prints a deprecation warning when the arguments match a main package.

In Go 1.18, go get will not be able to build or install packages. It will be dedicated to adjusting dependencies in go.mod.

When go get is run within a module and its arguments match a main package, the deprecation notice will no longer be printed. go get example.com/cmd is a fine way to upgrade a tool that the module depends on, and we don't want to discourage that.

When go get is run outside a module, it will print an error. There's no go.mod file to update, so it can't do anything other than populating the module cache. That might be occasionally useful, but it seems better to fail loudly, breaking in contexts where people haven't seen the deprecation warnings.

(All this only applies to go get in module-aware mode. GOPATH-mode is unaffected by this issue).

[gopherbot]

Change https://golang.org/cl/352150 mentions this issue: cmd/go: remove change 'go get -d' to 'go get' in tests

[gopherbot]

Change https://golang.org/cl/352149 mentions this issue: cmd/go: make 'go get' fail with an error when outside a module

[gopherbot]

Change https://golang.org/cl/352151 mentions this issue: cmd/go: add release note for 'go get' changes

[zikaeroh]

With this deprecated, what is the recommended way to install something like golang.org/x/tools/gopls@master (which has to be built with golang.org/x/tools@master at the same revision) without cloning the repo? See the instructions here: https://github.com/golang/tools/blob/master/gopls/doc/advanced.md#unstable-versions

go install doesn't have a way to specify multiple modules.

Forgive me if this has been brought up before (I can't recall if it has).

[jayconrod]

@zikaeroh This change is admittedly sacrificing flexibility in exchange for simplicity and predictability. go install example.com/cmd@version doesn't support changing versions of dependencies.

For gopls development, and in general when developing tools where a lot of changes are made across multiple modules, I expect the most convenient workflow will be to run go install . within the gopls directory. That will apply the local replace directive, referring to the parent module. Alternatively, a go.work file could be used (#45713).

As a user of a tool, if you want to upgrade dependencies beyond what the tool's go.mod asked for, you can do something like:

# Create an empty go.mod file, only for tracking requirements.
cd $(mktemp -d)
go mod init empty-module-for-installing-a-tool

# Use 'go get' to add requirements and to ensure they work together.
go get golang.org/x/tools/gopls@master golang.org/x/tools@master

# Install using the selected versions in go.mod.
go install golang.org/x/tools/gopls

[zikaeroh]

I see. That's effectively what I do now, I suppose (with https://github.com/zikaeroh/gotools, which does the above for gopls by converting its relative replacements to absolute replacements).

I was hoping to rewrite my tool manager to use go install to avoid having to build temporary go.mod + tools.go files for each binary (since nearly all of the time, there's no point; the tool's go.mod is plenty), but I suppose I can just deal with both.

I'm sure it's rare, I'm just one to always be testing gopls at the latest to report bugs as they appear, and prefer to not have to do something special to run it.

[jayconrod]

@zikaeroh Sorry for the extra trouble. I'm pretty sure we talked about this at some point in #40276, since it would have been easy to interpret arguments the same way go get does. I think we decided not to because it's not clear on the command line whether anything from golang.org/x/tools is actually installed. It might be a main package (installed), a regular package (compiled only? or not?) or a module that just acts as a version constraint. And you don't know which from the command line. With a query like @master it could change over time.

[zikaeroh]

No worries, I do agree that the ambiguity isn't worth it. I just also know that gopls is probably the most important tool I install these days, so wanted to bring it up (especially given official instructions say to use go get in a way that will stop working, and right now spits out an error).

It's not too bad for me to preserve the code that automates that mktemp -d; go mod init process whenever go install wouldn't be enough. Nearly all the knobs I have in GOBIN helper exist just to deal with gopls. 😄

[gopherbot]

Change https://golang.org/cl/354149 mentions this issue: cmd/go: do not check for a built binary in TestScript/mod_get_fossil

[heschi]

Is this done? What's left to do if not?

[jayconrod]

Implementation work is done. I still need to take a pass through the website and update any go get instructions to go install.

[gopherbot]

Change https://golang.org/cl/355209 mentions this issue: cmd/go: adjust documentation mentioning 'go get'

[gopherbot]

Change https://golang.org/cl/355249 mentions this issue: _content: change 'go get' to 'go install' where appropriate

[gopherbot]

Change https://golang.org/cl/355493 mentions this issue: cmd/go: stamp tags and flags in build info

[gopherbot]

Change https://golang.org/cl/357714 mentions this issue: all: update install commands to go install pkg@version