Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: suppress warning messages about symlinks #35941

Open
cespare opened this issue Dec 3, 2019 · 14 comments
Open

cmd/go: suppress warning messages about symlinks #35941

cespare opened this issue Dec 3, 2019 · 14 comments
Milestone

Comments

@cespare
Copy link
Contributor

@cespare cespare commented Dec 3, 2019

I'm converting a large source tree which used vendoring in GOPATH world into a Go module. An issue I've encountered is that go mod tidy and go mod why print some warnings about symlinks.

These look like:

$ go mod tidy
warning: ignoring symlink /path/to/some/symlink/dir
...

Where a warning line is printed for every symlinked directory in the repo.

These warnings make these 'go mod' commands unpleasant to use (you have to scroll through all the junk to see the relevant bits, if any, at the bottom).

None of the Go code is in symlinked directories. The symlinks are for other projects in other languages. For example, one use of symlinks is as a cheap way to share certain JS/CSS assets between multiple sub-projects.

Would it be possible to simply delete these warnings? Alternatively, could we remove the warning if the symlinked directory doesn't contain any Go code?

I tested this behavior in both Go 1.13.4 and tip (bf3ee57).

/cc @bcmills @jayconrod for cmd/go, and @rsc who originally wrote these warning messages as far as I can tell.

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Dec 3, 2019

It looks like this warning was introduced in CL 46425, long before modules. It's emitted any time a pattern is expanded (all or anything with ...) and a symlink to a directory is encountered. go mod tidy is doing that internally.

I don't think these warnings should be deleted entirely, but making them less spammy would be good. Not sure how to do that though. Checking for .go files in a symlinked directory is probably not enough, since the directory itself may not have any .go files, but it may have subdirectories that do. I think Kubernetes' vendor/k8s.io directory works like that (with symbolic links to staging). Traversing subdirectories after a symbolic link could get us into a loop though.

@jayconrod jayconrod added this to the Backlog milestone Dec 3, 2019
@cespare
Copy link
Contributor Author

@cespare cespare commented Dec 3, 2019

@jayconrod yeah, I've run into this in the past when doing things like go list ./....

However, I think the spam is more of a problem with modules:

  • go list is used a lot less in the day-to-day workflow in the GOPATH world than go mod commands are used in the modules world.
  • With go list ./... or similar, it's more understandable that you're asking the Go tool to please go discover all the packages. With go mod tidy it's less clear.

What about suppressing the warnings for go mod subcommands only? (And keeping them for go list?)

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Dec 3, 2019

Suppressing these warnings for go mod subcommands seems reasonable to me. @bcmills WDYT?

@bcmills
Copy link
Member

@bcmills bcmills commented Dec 4, 2019

If they're not appropriate for go mod, they're probably not appropriate for go list either. (And the converse: if they're useful for go list, then they're probably also useful for go mod.)

Given that folks can already explicitly prune out subdirectories using go.mod files, I'm curious about the directory structure that leads to this sort of situation. Are the non-Go symlinks in sibling directories to the Go source files, or in subdirectories of Go package directories?

As an alternative, would it make sense to do the go.mod pruning check before we do the symlink-pruning check?

@cespare
Copy link
Contributor Author

@cespare cespare commented Dec 5, 2019

If they're not appropriate for go mod, they're probably not appropriate for go list either. (And the converse: if they're useful for go list, then they're probably also useful for go mod.)

I explained above why I think it's reasonable to treat these differently, but if we have to treat them the same, then I'd err on the side of no spam.

Given that folks can already explicitly prune out subdirectories using go.mod files, I'm curious about the directory structure that leads to this sort of situation.

We have a large monorepo with projects in several languages. Before it was all a single GOPATH with vendored source. I'm converting it to be a single module instead.

Adding empty go.mod files to a bunch of project directories just to suppress this warning is probably a nonstarter.

Are the non-Go symlinks in sibling directories to the Go source files, or in subdirectories of Go package directories?

Currently, only sibling (or "cousin" or "cousin Nth removed") directories.

I didn't mind ignoring these warnings (or 2>/dev/null) when running go list in the GOPATH world. It just doesn't seem reasonable that I have to continue doing so with commands like go mod tidy, which is part of my normal workflow.

@agnivade
Copy link
Contributor

@agnivade agnivade commented Dec 5, 2019

Just to add a small point, we have a situation where a Go directory is linked with a symlink. (I know .. I know ..). So removing these warning would certainly help.

@bcmills
Copy link
Member

@bcmills bcmills commented Dec 5, 2019

Currently, only sibling (or "cousin" or "cousin Nth removed") directories.

So, could you move the go.mod file to the root of the Go source tree, instead of the repo root? (Or does it need access to testdata in sibling directories?)

@cespare
Copy link
Contributor Author

@cespare cespare commented Dec 5, 2019

So, could you move the go.mod file to the root of the Go source tree, instead of the repo root?

There is no single Go source tree. Everything's mixed. What I meant was only that no Go package directory or any descendent thereof contains symlinks.

├─go.mod
├─go.sum
├─go─project─1/
├─go─project─2/
├─non─go─project─1/
│ └─symlink
└─sub/
  ├─go─project─3/
  ├─non─go─project─2/
  │ └─symlink
  └─symlink
@sebd71
Copy link

@sebd71 sebd71 commented Jan 13, 2020

As workaround you can create an empty go.mod file in each directory you want to ignore.

touch no-go-project-1/go.mod
touch sub/no-go-project-2/go.mod

But this is not a perfect workaround because you ignore sub/symlink with this method.

See go wiki modules https://github.com/golang/go/wiki/Modules#can-an-additional-gomod-exclude-unnecessary-content-do-modules-have-the-equivalent-of-a-gitignore-file

@cespare
Copy link
Contributor Author

@cespare cespare commented Jan 13, 2020

@sebd71 as I wrote above:

Adding empty go.mod files to a bunch of project directories just to suppress this warning is probably a nonstarter.

It's simply not reasonable for us to add lots of go.mod files to non-Go projects, especially in an ongoing way (adding and deleting them as we add and delete symlinks).

@sebd71
Copy link

@sebd71 sebd71 commented Jan 13, 2020

I'm agree with you and that's why I said that it's not a perfect workaround.
I'm also facing the same problem.
A good solution would be to add in root go.mod "exclude mydir/*" statement, but unfortunately it seems not supported for now.

@marceloneil marceloneil mentioned this issue May 6, 2020
36 of 40 tasks complete
@cespare
Copy link
Contributor Author

@cespare cespare commented Jun 2, 2020

@bcmills @jayconrod would you accept a PR for this once the tree opens? We still haven't migrated our code to modules and this tooling wart is definitely a sticking point.

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Jun 4, 2020

Discussed with @matloob, @bcmills offline. It seems like the best thing to do here is only print this warning when there's an explicit argument that has a ..., and the go command does not follow a symbolic link to a directory when expanding that pattern. The warning should not be printed for all, nor should it be printed for go mod tidy.

@cespare Does that work for you?

I'd be happy to review a CL for this. We can review any time, but it won't be merged until the tree opens for 1.16.

@cespare
Copy link
Contributor Author

@cespare cespare commented Jun 4, 2020

@cespare Does that work for you?

Absolutely. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants