Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proposal: global ignore mechanism for Go tool ecosystem #42965

Open
burdiyan opened this issue Dec 3, 2020 · 9 comments
Open

proposal: global ignore mechanism for Go tool ecosystem #42965

burdiyan opened this issue Dec 3, 2020 · 9 comments
Labels
Projects
Milestone

Comments

@burdiyan
Copy link

@burdiyan burdiyan commented Dec 3, 2020

Problem

For non-trivial (often multi-language) projects it's often desirable to make all the Go tools (including gopls) ignore certain directories.

Some of the examples could be the huge amount of files within node_modules, or bazel-* directories generated by Bazel. This causes many operations with ./... wildcards taking longer than desired. Also gopls often eats up a lot of CPU in VS Code depending on what you are doing.

Prior Art

This is something that has been discussed in several issues before, but seems like people couldn't get agree on a solution.

Some tools started to have their own solutions which causes fragmentation and is cumbersome.

For example goimports have its own machinery for this - .goimportsignore file in this case. But it's not working with Go Modules.

Other tools have a hard-coded list of directories to ignore, like .git and so on.

It seems like having a global solution that all the Go ecosystem could understand would make sense to solve this kind of problem.

Recently a workaround for this was to place a dummy go.mod file in the directories you wanted to ignore. But this is not easily portable between users of the project, because often these directories can be re-created on the user's machine and aren't even checked-in. Asking people to sprinkle some go.mod files all around every time is cumbersome.

@robpike was against of creating more dot files (#30058 (comment)).

Proposed Solution

I'm proposing to add this configuration into the existing go.mod file. Maybe extending the existing exclude directive to support file paths, or creating a new directive for this purpose.

Another solution could be a global .goignore file. This would go against Rob's desire to avoid new dot files, but would be in the spirit with other tools like that have files like .dockerignore, .gitignore, .bazelignore, etc.

/cc @tj @stamblerre

@gopherbot gopherbot added this to the Proposal milestone Dec 3, 2020
@mvdan
Copy link
Member

@mvdan mvdan commented Dec 3, 2020

But this is not easily portable between users of the project, because often these directories can be re-created on the user's machine and aren't even checked-in. Asking people to sprinkle some go.mod files all around every time is cumbersome.

I'm not sure that I understand this argument. Presumably, it's a program that creates and fills those directories, since they have to contain a significant amount of files for you to really want to ignore them in Go. If they were just a handful of files created manually by a human, it would be a negligible cost for Go to walk those and realise there are no Go packages there.

So, given that it is a program or script creating those large directories, why not add a touch ${dir}/go.mod at the end? That seems easy enough at a high level, at least.

I'm proposing to add this configuration into the existing go.mod file.

This is unlikely to happen, see #42343 (comment).

Another solution could be a global .goignore file. This would go against Rob's desire to avoid new dot files, but would be in the spirit with other tools like that have files like .dockerignore, .gitignore, .bazelignore, etc.

I have to admit that I dislike this option. It's bad enough that all these other tools use separate ignore files.

@burdiyan
Copy link
Author

@burdiyan burdiyan commented Dec 3, 2020

I'm not sure that I understand this argument. Presumably, it's a program that creates and fills those directories, since they have to contain a significant amount of files for you to really want to ignore them in Go. If they were just a handful of files created manually by a human, it would be a negligible cost for Go to walk those and realise there are no Go packages there.

So, given that it is a program or script creating those large directories, why not add a touch ${dir}/go.mod at the end? That seems easy enough at a high level, at least.

@mvdan It is indeed a program that creates these directories. But it's a program that you don't control normally. Wrapping well-known tools like nom install with your own script only to put an empty go.mod in there doesn't seem right.

On the other hand by placing arbitrary files in these directories you're invading the territory of other tools. What if that program checks the integrity of the directory and would break seeing a random unknown file? It's not the case with node_modules but breaking into structures created by other programs, only to work around your own problem doesn't seem right either.

I understand the objection about go.mod. I was not aware about @rsc's statement.

I have to admit that I dislike this option. It's bad enough that all these other tools use separate ignore files.

Could you elaborate on why do you think it's bad? It may not be the most elegant solution, but it's common practice, well-understood and somewhat expected.

If we already have .goimportsignore, why not standardizing it into something that can be handled and understood by all the ecosystem of Go tools?

@bcmills
Copy link
Member

@bcmills bcmills commented Dec 3, 2020

It is indeed a program that creates these directories. But it's a program that you don't control normally. Wrapping well-known tools like nom install with your own script only to put an empty go.mod in there doesn't seem right.

Wouldn't you need to wrap the tool to add a .goignore file anyway? (Given that you need to inject a file, why does it matter whether it is named .goignore or go.mod?)

@burdiyan
Copy link
Author

@burdiyan burdiyan commented Dec 3, 2020

@bcmills My proposal is to add a file in the root of the project, not in the directory being ignored. So it would be checked in. Like .gitignore in Git. Basically the idea is to list the paths to ignore in that file, and check it in.

@bcmills
Copy link
Member

@bcmills bcmills commented Dec 3, 2020

...ok? But why would you not also check in the injected go.mod files?

@mvdan
Copy link
Member

@mvdan mvdan commented Dec 3, 2020

Could you elaborate on why do you think it's bad? It may not be the most elegant solution, but it's common practice, well-understood and somewhat expected.

The common practice is to litter repositories with dot files. That does not mean we should do the same, making the problem worse :) Go already has multiple mechanisms to ignore entire directories (. or _ prefixes, and dropping empty go.mod files), so there needs to be a really good reason to add another method.

@burdiyan
Copy link
Author

@burdiyan burdiyan commented Dec 3, 2020

...ok? But why would you not also check in the injected go.mod files?

@bcmills because often directories to ignore aren't checked in.

@ianlancetaylor ianlancetaylor added this to Incoming in Proposals Dec 3, 2020
@burdiyan
Copy link
Author

@burdiyan burdiyan commented Dec 4, 2020

The common practice is to litter repositories with dot files. That does not mean we should do the same, making the problem worse :) Go already has multiple mechanisms to ignore entire directories (. or _ prefixes, and dropping empty go.mod files), so there needs to be a really good reason to add another method.

@mvdan IMHO, having a dot file in one place, that is trackable, is less of an evil, than sprinkling empty go.mod files all over the place, ad-hoc, and breaking into opinions of other tools.

Thinking about pros and cons of implementing such a feature, I'm struggling to see any cons (probably due to my ignorance), besides having to spend the time to implement it. I'd appreciate if anyone could bring some light into this to understand the implications.

@psigen
Copy link

@psigen psigen commented Mar 19, 2021

I have a use case where I have a multi-language repo where not all of the developers are touching the go components.

I don't think it is reasonable to ask my docker, bazel, and nodejs developers to all wrap their normal tooling in scripts that touch extra files in their build directories, nor ask them to try to rename their standard build directories to match existing go conventions, some of which conflict with the other tool conventions.

It seems like there should be a way to specify how to ignore certain files or directories that does not require modifying the content of those files or directories, because the ignored content is not being managed by go and may have its own conflicting conventions and lifecycle.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Proposals
Incoming
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants