Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: enable listing direct dependency updates #40364

Open
icholy opened this issue Jul 23, 2020 · 18 comments
Open

cmd/go: enable listing direct dependency updates #40364

icholy opened this issue Jul 23, 2020 · 18 comments

Comments

@icholy
Copy link

@icholy icholy commented Jul 23, 2020

Intro

The recommended approach to listing available module updates is to run the following command.

go list -u -m all

This displays every dependency along with newer available versions.

go list -u -m all
github.com/icholy/utm
github.com/google/go-cmp v0.5.0 [v0.5.1]
github.com/pkg/errors v0.8.1 [v0.9.1]
github.com/spf13/pflag v1.0.3 [v1.0.5]
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2 [v0.0.0-20200709230013-948cd5f35899]
golang.org/x/net v0.0.0-20190311183353-d8887717615a [v0.0.0-20200707034311-ab3426394381]
golang.org/x/sync v0.0.0-20190423024810-112230192c58 [v0.0.0-20200625203802-6e8e738ad208]
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a [v0.0.0-20200722175500-76b94024e4b6]
golang.org/x/text v0.3.0 [v0.3.3]
golang.org/x/tools v0.0.0-20190624222133-a101b041ded4 [v0.0.0-20200723000907-a7c6fd066f6d]
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543
gotest.tools/v3 v3.0.2

My go.mod only specifies two dependencies.

  
module github.com/icholy/utm

go 1.14

require (
	github.com/google/go-cmp v0.5.0
	gotest.tools/v3 v3.0.2
)

Problem

The go list output gets completely unmanageable when the dependency tree grows. When listing available module updates, I'm only interested in my direct dependencies. Those are the only ones I usually have control over.

The go list command has a -json flag which makes it possible to feed into external tools such as https://github.com/psampaz/go-mod-outdated. However, I find it strange that I need external tooling to ask "which modules in go.mod are outdated"

Proposal

I propose a mechanism for limiting the go list output to direct dependencies.

go list -u -m direct

Relates to #40323

@gopherbot gopherbot added this to the Proposal milestone Jul 23, 2020
@gopherbot gopherbot added the Proposal label Jul 23, 2020
@mvdan
Copy link
Member

@mvdan mvdan commented Jul 23, 2020

We've discussed this multiple times before with the tooling/modules folks, mostly on Slack and on the golang-tools monthly calls. The only written down notes I could find right now are #27887 (comment); @bcmills @rogpeppe @jayconrod @myitcv might have more links.

@icholy
Copy link
Author

@icholy icholy commented Jul 23, 2020

The provided solution uses go lists template ability to filter out indirect dependencies.

go list -f '{{if not .Indirect}}{{.}}{{end}}' -u -m all

I don't think it's reasonable to expect users to type this out every time. A bash alias can be created, but that's not portable and has the same problems as using an external tool (you have to know about it, and set it up).

The common use-case for people developing module tooling is not the same as for people consuming the module tooling. I think that most people just want to know if they have any dependencies to go get -u. Usually in Go, simple thing are simple to do, but complex things are still possible. go list only gives us that second half.

edit: I suppose what I actually want is a veneer on top of go list (ex: go mod updates).

@icholy icholy changed the title Proposal: cmd/go Enable only listing direct dependencies Proposal: cmd/go Enable only listing direct dependency updates Jul 23, 2020
@icholy icholy changed the title Proposal: cmd/go Enable only listing direct dependency updates Proposal: cmd/go Enable listing direct dependency updates Jul 23, 2020
@jayconrod jayconrod changed the title Proposal: cmd/go Enable listing direct dependency updates proposal: cmd/go: enable listing direct dependency updates Jul 23, 2020
@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Jul 23, 2020

I think this would be good to have. It's pretty common to want to list or upgrade direct dependencies of packages in the main module. It's too hard to do that with go list right now. Some thoughts though:

  • go get -d direct should work, too, not just go list -m -u direct.
  • We need a clear definition of direct. When lazy loading is implemented (#36460), the set of modules required in go.mod will expand, so I don't think it would make sense to define direct module dependencies as those required in go.mod. Even today, there may be // indirect requirements, and those dependencies may not be listed anywhere else. Maybe it should be defined at the package level as "the set of packages directly imported by packages in the main module".
@ianlancetaylor ianlancetaylor added this to Incoming in Proposals Aug 7, 2020
@icholy
Copy link
Author

@icholy icholy commented Aug 28, 2020

An alternative to direct is imported.

go list -m -u imported

@rsc rsc moved this from Incoming to Active in Proposals Sep 23, 2020
@thepudds
Copy link

@thepudds thepudds commented Sep 24, 2020

Some overlap here with #28424.

@rsc
Copy link
Contributor

@rsc rsc commented Sep 30, 2020

Thanks @thepudds. Sorry we missed discussing that one. What do you think about this proposal vs the other?

@rsc
Copy link
Contributor

@rsc rsc commented Oct 7, 2020

/cc @jayconrod @bcmills; what's the status of this?

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Oct 7, 2020

cc @matloob

I think the idea is good, but I'm not sure about the CLI specifics.

  • The direct keyword seems a little terse. I'm not sure go list direct will make sense to someone unfamiliar with it. Any other keyword suggestions?
  • I'd prefer direct over imports, since we'd want direct to cover both package and module dependencies, depending on whether -m is used.
  • Would go list -m direct just list requirements in go.mod? That won't be all that useful after lazy loading (#36460) because go.mod will list all modules needed to build packages in the main module, not just those providing directly imported packages.
  • direct seems like more of a function than a set. In this proposal, we're talking about direct dependencies of the main module, but we might want to be able to get the direct dependencies of another set of packages: go list direct(example.com/m/...). Personally, I'd love to have a query language like bazel query, with functions like rdeps and somepath. But that's certainly overcomplicating things. If we eventually did that, direct on its own could still imply the main module or its packages.
@bcmills
Copy link
Member

@bcmills bcmills commented Oct 7, 2020

The direct keyword seems a little terse.

Heh, I had a bit of the opposite reaction! All of the existing meta-packages (all, std, cmd) are three letters, so I was kind of hoping we could find another suitable three-letter pattern. (But I can't think of one at the moment.)

@bcmills
Copy link
Member

@bcmills bcmills commented Oct 7, 2020

Would go list -m direct just list requirements in go.mod?

I think it should list only those requires that are not labeled with an // indirect comment.
That admittedly assigns semantic meaning (as an input) to comments that were previously only an output (produced, but not semantically consumed by, the go command), but I think it's more in line with user intent for the direct keyword.

Or, we could disallow the direct pattern with the -m flag, although that would arguably make the direct pattern less useful.

@bcmills
Copy link
Member

@bcmills bcmills commented Oct 7, 2020

direct seems like more of a function than a set.

I agree that it would be nice for go list to support querying “the packages directly imported by the named packages”, but I think that feature should be mostly orthogonal to the one proposed here.

(I think the hypothesized go list query is more closely related to the -deps flag than to the all pattern.)

@icholy
Copy link
Author

@icholy icholy commented Oct 7, 2020

Or, we could disallow the direct pattern with the -m flag, although that would arguably make the direct pattern less useful.

Using it with -m is my primary use-case.

@rsc
Copy link
Contributor

@rsc rsc commented Oct 21, 2020

It sounds like people are saying:

  • In go list direct, direct expands to a list of all the packages directly imported by the current module.
  • In go list -m direct, direct expands to a list of all the modules providing those packages.

That is, direct becomes a pseudo-name like std and cmd and all that expands to this meaning in all commands, not just go list. But like with go list -m all, the meaning of direct changes for go list -m (as it must, since -m is about modules, not packages).

Do I have that right? Does anyone object to that?

@icholy
Copy link
Author

@icholy icholy commented Oct 21, 2020

Re #28424

It seems that the direct pseudo-name should also work with go get.

go get -u direct
@rsc
Copy link
Contributor

@rsc rsc commented Oct 28, 2020

Based on the discussion, this seems like a likely accept.

@rsc rsc moved this from Active to Likely Accept in Proposals Oct 28, 2020
@rsc
Copy link
Contributor

@rsc rsc commented Oct 28, 2020

(If accepted, this would not be until Go 1.17. There's already a lot for Go 1.16.)

@mvdan
Copy link
Member

@mvdan mvdan commented Oct 29, 2020

In go list -m direct, direct expands to a list of all the modules providing those packages.

Just so I understand, this would be a subset of the modules listed in go.mod without // indirect after a go mod tidy, right? Since lazy module loading can also add lines to go.mod for indirect modules without // indirect as far as I remember.

@rsc
Copy link
Contributor

@rsc rsc commented Nov 4, 2020

No change in consensus, so accepted.

@rsc rsc moved this from Likely Accept to Accepted in Proposals Nov 4, 2020
@rsc rsc changed the title proposal: cmd/go: enable listing direct dependency updates cmd/go: enable listing direct dependency updates Nov 4, 2020
@rsc rsc modified the milestones: Proposal, Backlog Nov 4, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Proposals
Accepted
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
7 participants