Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: most 'go mod' subcommands should not update go.mod, go.sum #45551

Open
jayconrod opened this issue Apr 13, 2021 · 4 comments
Open

cmd/go: most 'go mod' subcommands should not update go.mod, go.sum #45551

jayconrod opened this issue Apr 13, 2021 · 4 comments
Milestone

Comments

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Apr 13, 2021

As of Go 1.16, all go mod subcommands can update go.mod and go.sum. Some commands are intended to update these files, and that shouldn't change. However, most commands may still perform an automatic update, which may be surprising. Internally, go mod subcommands run in -mod=mod mode, even though they don't have a -mod command line flag.

For some commands, updating go.mod and go.sum is expected:

  • go mod edit is supposed to edit go.mod.
  • go mod init creates go.mod and may import information from another dependency management tool.
  • go mod tidy is supposed to edit go.mod and go.sum.

For other commands, updating go.mod or go.sum may be quite unexpected. The go command should report an error if go.mod or go.sum need to be updated.

  • go mod graph is a query tool. It shouldn't have side effects.
  • go mod vendor is expected to update vendor but not go.mod.
  • go mod verify also shouldn't have side effects.
  • go mod why, same deal.

go mod download occupies an awkward middle ground. It has always been able to update go.mod (probably should not), but it may also add sums of modules in the build list to go.sum. This is usually unexpected and undesired when go mod download is invoked without arguments: users want something like go list all that populates the module cache with modules needed to build packages in the main module, not including unused dependencies of dependencies. However, we shouldn't entirely prevent go mod download from updating go.sum, since it's the best way to add a specific sum for a necessary but unused module (for disambiguating an imported package).

I'd suggest the following change in behavior for go mod download:

  • go mod download always loads the build list and reports an error if go.mod needs to be updated instead of updating it automatically.
  • go mod download only adds a hash to go.sum if the corresponding module is explicitly named on the command line (not simply matched by all or a wildcard or implied by a lack of arguments) without an explicit version or at the version in the build list.

cc @bcmills @matloob

@jayconrod jayconrod added this to the Backlog milestone Apr 13, 2021
@zikaeroh
Copy link
Contributor

@zikaeroh zikaeroh commented Apr 13, 2021

Out of curiosity, would this proposal change the behavior of go mod download when go.mod says one thing, but the code says another? I'm not personally clear what "build list" is defined as, but IIRC it may be determined by the code present in the module.

My main use case for go mod download is docker image caching, where I often do something like:

COPY ./go.mod ./go.sum ./
RUN go mod download

COPY ./ ./

RUN go build

Such that I can download the dependencies given in go.mod into a layer, then bring the code in; that way when I modify the code my module cache is preserved and I don't need to rebuild the image.

In this case, I cannot use go list all because that requires the code to be present, meaning that when the code is changed, docker invalidates the layer (as the files changed), and the module cache is gone.

@jayconrod
Copy link
Contributor Author

@jayconrod jayconrod commented Apr 13, 2021

@zikaeroh I think this proposal would improve things for you, at least a bit.

To clarify, go mod download doesn't read code, either before or after this proposal. When invoked without arguments or with the argument all, it downloads all the modules in the build list (the set of modules and versions selected by MVS). This is the same set of modules shown by go list -m all.

go mod download without arguments generally downloads modules you don't need (#41431). This proposal won't change that, but lazy loading (#36460) should help.

The difference here is that go mod download without arguments would not write go.sum, so you won't get a bunch unwanted changes there.

@zikaeroh
Copy link
Contributor

@zikaeroh zikaeroh commented Apr 13, 2021

To clarify, go mod download doesn't read code, either before or after this proposal. When invoked without arguments or with the argument all, it downloads all the modules in the build list (the set of modules and versions selected by MVS). This is the same set of modules shown by go list -m all.

Thanks for confirming. I'm alright with even a bad version of go mod download that over-downloads, since downloading some 2GB of dependencies once is better than redownloading 1GB every time I do a build. 🙂

@gopherbot
Copy link

@gopherbot gopherbot commented Apr 14, 2021

Change https://golang.org/cl/310073 mentions this issue: cmd/go: explicitly set cfg.BuildMod in commands without -mod flag

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants