Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: deprecate installing binaries using 'go get' in Go 1.17 and make 'go get -d' the default behavior #43684

Closed
bcmills opened this issue Jan 14, 2021 · 46 comments
Labels
NeedsFix release-blocker
Milestone

Comments

@bcmills
Copy link
Member

@bcmills bcmills commented Jan 14, 2021

Proposal #40276 added a versioned go install variant that works outside of a module.

As part of the changes approved in that proposal, we plan to deprecate the use of the go get to install binaries, and make go get -d (which downloads source code for but does not build the requested packages) the default behavior.

We had planned to warn about the use of go get to install binaries in Go 1.16 and make it the default in Go 1.17. However, in #42885 we decided to also delay the warning until Go 1.17, so that third-party projects that support the most recent two major Go releases (as the Go project itself does) can give users a single non-deprecated install command, rather than a confusing menu of commands that vary by Go version.

I don't see an issue filed yet to track that change in Go 1.17, so this is that tracking issue. (CC @jayconrod @matloob)

@bcmills bcmills added this to the Go1.17 milestone Jan 14, 2021
@bcmills bcmills added early-in-cycle release-blocker NeedsFix labels Jan 14, 2021
@bcmills
Copy link
Member Author

@bcmills bcmills commented Jan 14, 2021

It is not clear to me whether we should print a deprecation notice in Go 1.17 and change the default behavior in Go 1.18, or change the default behavior in 1.17 and merely print a notice informing the user of the change if their go get pattern happens to match a package main.

@ianlancetaylor

This comment has been minimized.

@bcmills

This comment has been minimized.

@andig
Copy link
Contributor

@andig andig commented Jan 14, 2021

As part of the changes approved in that proposal, we plan to deprecate the use of the go get to install binaries, and make go get -d (which downloads source code for but does not build the requested packages) the default behavior.

I‘m all for this. Its a regular source of confusion that go get fails when I pull the sources for later cross-compile and the go get fails with architecture-specific error messages.

@lu4p

This comment has been minimized.

@bcmills

This comment has been minimized.

@gopherbot
Copy link

@gopherbot gopherbot commented Feb 24, 2021

This issue is currently labeled as early-in-cycle for Go 1.17.
That time is now, so a friendly reminder to look at it again.

@ohir
Copy link

@ohir ohir commented Feb 27, 2021

change the default behavior in 1.17 and merely print a notice informing the user of the change if their go get pattern happens to match a package main.

The sooner the better. Just add a short instruction for the non-Go user stumbling upon older source that "Since Go 1.17 to install an app from the source you need to use go install instead of go get". I'd also suggest adding the -u flag to the install - silent or with a compact message that "-u is not needed with install".

@dmitshur
Copy link
Contributor

@dmitshur dmitshur commented Mar 25, 2021

Friendly ping @bcmills, @jayconrod, @matloob since this is a release blocking issue and has an early in cycle label.

@jayconrod jayconrod self-assigned this Mar 25, 2021
@gopherbot
Copy link

@gopherbot gopherbot commented Mar 30, 2021

Change https://golang.org/cl/305670 mentions this issue: cmd/go: print deprecation notice for 'go get cmd'

@gopherbot
Copy link

@gopherbot gopherbot commented Apr 5, 2021

Change https://golang.org/cl/305649 mentions this issue: _content/doc: add a page explaining 'go get' deprecation

gopherbot pushed a commit to golang/website that referenced this issue Apr 5, 2021
'go get' will link here when the arguments refer to main packages and
the -d flag is not enabled.

For golang/go#43684

Change-Id: I62045a4451bb767a83d9401665420b164c6ca255
Reviewed-on: https://go-review.googlesource.com/c/website/+/305649
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
gopherbot pushed a commit that referenced this issue Apr 5, 2021
The notice is shown when 'go get' is invoked with the -d flag, and
the arguments match at least one main package.

This reverts CL 274552.

For #43684

Change-Id: I42e6731455f22988bf72dde1d5a76d197e9e3954
Reviewed-on: https://go-review.googlesource.com/c/go/+/305670
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
@fzipp
Copy link
Contributor

@fzipp fzipp commented Apr 8, 2021

If installing binaries using 'go get' is deprecated, the Go documentation should lead by example and use the recommended 'go install' command. There are:

  • contribute.html
    go get -u golang.org/x/review/git-codereview
    go get -u golang.org/x/tools/cmd/go-contrib-init
    go get -u golang.org/x/review/git-codereview

  • docs.html
    go get golang.org/x/tour

  • install-source.html
    go get golang.org/x/tools/cmd/godoc (x2)

  • manage-install.html
    go get golang.org/dl/go1.10.7

Also, many README files in the Go repositories instruct the user to install binaries using 'go get':

https://github.com/golang/blog
https://github.com/golang/dl
https://github.com/golang/example
https://github.com/golang/lint
https://github.com/golang/perf
https://github.com/golang/review
https://github.com/golang/tools
https://github.com/golang/tour
https://github.com/golang/appengine

@toothrot
Copy link
Contributor

@toothrot toothrot commented Apr 22, 2021

@bcmills We're getting close to the freeze. Just checking in as this issue is both early-in-cycle and a release-blocker. Any updates?

@bcmills bcmills added okay-after-beta1 and removed early-in-cycle labels Apr 23, 2021
@bcmills
Copy link
Member Author

@bcmills bcmills commented Apr 23, 2021

Jay implemented the deprecation notice in CL 305670. I believe we're planning to actually change the default behavior in 1.18, so I think we're done with the 1.17 parts of this.)

(I'd like to confirm that understanding with Jay after he gets back, though. Leaving as release-blocker and marking okay-after-beta1 to make sure we do that.)

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented May 3, 2021

@bcmills comment is correct.

I still need to add a release note and update documentation for 1.17. After that, we can re-milestone to 1.18 for the default behavior change.

@gopherbot
Copy link

@gopherbot gopherbot commented May 10, 2021

Change https://golang.org/cl/318570 mentions this issue: cmd/go: don't print a 'go get' deprecation notices in the main module

@pellared
Copy link

@pellared pellared commented May 12, 2021

Is not the go get approach safer? Does it not validate the modules against go.sum? For sure it is bypassed when using go install.

If what I suspect is true then usage of go install is more vulnerable to supply chain attacks.

@gopherbot
Copy link

@gopherbot gopherbot commented Sep 15, 2021

Change https://golang.org/cl/349997 mentions this issue: cmd/go: 'go get' no longer builds or installs packages

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Sep 24, 2021

It is not clear to me whether we should print a deprecation notice in Go 1.17 and change the default behavior in Go 1.18, or change the default behavior in 1.17 and merely print a notice informing the user of the change if their go get pattern happens to match a package main.

In Go 1.17, go get without -d prints a deprecation warning when the arguments match a main package.

In Go 1.18, go get will not be able to build or install packages. It will be dedicated to adjusting dependencies in go.mod.

When go get is run within a module and its arguments match a main package, the deprecation notice will no longer be printed. go get example.com/cmd is a fine way to upgrade a tool that the module depends on, and we don't want to discourage that.

When go get is run outside a module, it will print an error. There's no go.mod file to update, so it can't do anything other than populating the module cache. That might be occasionally useful, but it seems better to fail loudly, breaking in contexts where people haven't seen the deprecation warnings.

(All this only applies to go get in module-aware mode. GOPATH-mode is unaffected by this issue).

@gopherbot
Copy link

@gopherbot gopherbot commented Sep 24, 2021

Change https://golang.org/cl/352150 mentions this issue: cmd/go: remove change 'go get -d' to 'go get' in tests

@gopherbot
Copy link

@gopherbot gopherbot commented Sep 24, 2021

Change https://golang.org/cl/352149 mentions this issue: cmd/go: make 'go get' fail with an error when outside a module

@gopherbot
Copy link

@gopherbot gopherbot commented Sep 24, 2021

Change https://golang.org/cl/352151 mentions this issue: cmd/go: add release note for 'go get' changes

gopherbot pushed a commit that referenced this issue Sep 28, 2021
As part of #40267, 'go install' is now fully responsible for building
and installing executables. 'go get' will only be used to change
versions in go.mod. The -d flag no longer has any effect; its behavior
is the default.

When 'go get' is invoked inside a module on a main package outside of
the main module, it no longer prints any warning. In 1.16-1.17, we
suggested using -d in this situation, but we want
'go get example.com/cmd' to be able to upgrade a tool dependency
without needing -d to suppress the warning.

For #43684

Change-Id: I9daf29c123a5a0e382aa326d62721cb26fc26c19
Reviewed-on: https://go-review.googlesource.com/c/go/+/349997
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
gopherbot pushed a commit that referenced this issue Sep 28, 2021
There's no go.mod file for 'go get' to update, so it has no effect,
other than checking arguments and filling the module cache. That might
be useul in some cases, but it seems better to fail loudly in case the
user hasn't seen the deprecation warning, for example, inside a
script.

For #43684

Change-Id: I6e67c782e3a1cb7046eac5c9df17eda7a31c7bce
Reviewed-on: https://go-review.googlesource.com/c/go/+/352149
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
gopherbot pushed a commit that referenced this issue Sep 28, 2021
The -d flag has no effect in module mode.

GOPATH tests are left alone.

For #43684

Change-Id: If0f0aad73d8b543ca4058fe9c9fea9d7fd7f95bd
Reviewed-on: https://go-review.googlesource.com/c/go/+/352150
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
gopherbot pushed a commit that referenced this issue Sep 28, 2021
For #43684

Change-Id: I9ce47de82203ec87e7d3683f56e6c6d61ae255f5
Reviewed-on: https://go-review.googlesource.com/c/go/+/352151
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
@zikaeroh
Copy link
Contributor

@zikaeroh zikaeroh commented Sep 28, 2021

With this deprecated, what is the recommended way to install something like golang.org/x/tools/gopls@master (which has to be built with golang.org/x/tools@master at the same revision) without cloning the repo? See the instructions here: https://github.com/golang/tools/blob/master/gopls/doc/advanced.md#unstable-versions

go install doesn't have a way to specify multiple modules.

Forgive me if this has been brought up before (I can't recall if it has).

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Sep 28, 2021

@zikaeroh This change is admittedly sacrificing flexibility in exchange for simplicity and predictability. go install example.com/cmd@version doesn't support changing versions of dependencies.

For gopls development, and in general when developing tools where a lot of changes are made across multiple modules, I expect the most convenient workflow will be to run go install . within the gopls directory. That will apply the local replace directive, referring to the parent module. Alternatively, a go.work file could be used (#45713).

As a user of a tool, if you want to upgrade dependencies beyond what the tool's go.mod asked for, you can do something like:

# Create an empty go.mod file, only for tracking requirements.
cd $(mktemp -d)
go mod init empty-module-for-installing-a-tool

# Use 'go get' to add requirements and to ensure they work together.
go get golang.org/x/tools/gopls@master golang.org/x/tools@master

# Install using the selected versions in go.mod.
go install golang.org/x/tools/gopls

@zikaeroh
Copy link
Contributor

@zikaeroh zikaeroh commented Sep 28, 2021

I see. That's effectively what I do now, I suppose (with https://github.com/zikaeroh/gotools, which does the above for gopls by converting its relative replacements to absolute replacements).

I was hoping to rewrite my tool manager to use go install to avoid having to build temporary go.mod + tools.go files for each binary (since nearly all of the time, there's no point; the tool's go.mod is plenty), but I suppose I can just deal with both.

I'm sure it's rare, I'm just one to always be testing gopls at the latest to report bugs as they appear, and prefer to not have to do something special to run it.

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Sep 28, 2021

@zikaeroh Sorry for the extra trouble. I'm pretty sure we talked about this at some point in #40276, since it would have been easy to interpret arguments the same way go get does. I think we decided not to because it's not clear on the command line whether anything from golang.org/x/tools is actually installed. It might be a main package (installed), a regular package (compiled only? or not?) or a module that just acts as a version constraint. And you don't know which from the command line. With a query like @master it could change over time.

@zikaeroh
Copy link
Contributor

@zikaeroh zikaeroh commented Sep 28, 2021

No worries, I do agree that the ambiguity isn't worth it. I just also know that gopls is probably the most important tool I install these days, so wanted to bring it up (especially given official instructions say to use go get in a way that will stop working, and right now spits out an error).

It's not too bad for me to preserve the code that automates that mktemp -d; go mod init process whenever go install wouldn't be enough. Nearly all the knobs I have in GOBIN helper exist just to deal with gopls. 😄

@gopherbot
Copy link

@gopherbot gopherbot commented Oct 5, 2021

Change https://golang.org/cl/354149 mentions this issue: cmd/go: do not check for a built binary in TestScript/mod_get_fossil

gopherbot pushed a commit that referenced this issue Oct 5, 2021
This test hasn't passed since CL 349997, but the failure was not
detected because the Go project's builders do not have a 'fossil'
binary installed (#48802).

For #43684

Change-Id: I25544574ab48f4f146ae3795e541179e78815758
Reviewed-on: https://go-review.googlesource.com/c/go/+/354149
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
@heschi
Copy link
Contributor

@heschi heschi commented Oct 6, 2021

Is this done? What's left to do if not?

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Oct 6, 2021

Implementation work is done. I still need to take a pass through the website and update any go get instructions to go install.

@gopherbot
Copy link

@gopherbot gopherbot commented Oct 11, 2021

Change https://golang.org/cl/355209 mentions this issue: cmd/go: adjust documentation mentioning 'go get'

@gopherbot
Copy link

@gopherbot gopherbot commented Oct 11, 2021

Change https://golang.org/cl/355249 mentions this issue: _content: change 'go get' to 'go install' where appropriate

gopherbot pushed a commit to golang/website that referenced this issue Oct 12, 2021
For golang/go#43684

Change-Id: Ifdb0695d15961150960f7be3eb0fb44ac1f0d4d2
Reviewed-on: https://go-review.googlesource.com/c/website/+/355249
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
gopherbot pushed a commit that referenced this issue Oct 12, 2021
In module-aware mode, 'go get' no longer builds or installs packages.

- 'go generate' explains build commands do not run generate
  commands. 'go get' is no longer a build command, so this CL removes
  mention of it.
- 'go get' will continue to accept build flags, but they're
  ignored. The documentation no longer mentions them, though it does
  mention -x for printing VCS commands.

For #43684

Change-Id: I1eea7241eecf72ba9f98238b729d91cc77ec7665
Reviewed-on: https://go-review.googlesource.com/c/go/+/355209
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
@gopherbot
Copy link

@gopherbot gopherbot commented Oct 12, 2021

Change https://golang.org/cl/355493 mentions this issue: cmd/go: stamp tags and flags in build info

@gopherbot
Copy link

@gopherbot gopherbot commented Oct 21, 2021

Change https://golang.org/cl/357714 mentions this issue: all: update install commands to go install pkg@version

gopherbot pushed a commit to golang/tools that referenced this issue Oct 27, 2021
Updates golang/go#43684
Updates golang/go#49101

Change-Id: I2f0c7920bcd6ce0429f1b7bdda3747bf376775f2
Reviewed-on: https://go-review.googlesource.com/c/tools/+/357714
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
Trust: Rebecca Stambler <rstambler@golang.org>
Trust: Peter Weinberger <pjw@google.com>
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsFix release-blocker
Projects
None yet
Development

No branches or pull requests