Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/tools/go/packages: can't distinguish dynamically-generated sources from checked-in ones #27909

bcmills opened this issue Sep 27, 2018 · 3 comments


Copy link

@bcmills bcmills commented Sep 27, 2018

When using go list -test, we can distinguish user-provided sources from generated ones based on whether the paths are absolute, as documented here (emphasis mine):

By default, the lists GoFiles, CgoFiles, and so on hold names of files in Dir (that is, paths relative to Dir, not absolute paths). The generated files added when using the -compiled and -test flags are absolute paths referring to cached copies of generated Go source files. Although they are Go source files, the paths may not end in ".go".

Unfortunately, removes that distinction by making all of the GoFiles it returns absolute, without apparently preserving enough information to undo that transformation. That makes it very difficult to use go/packages to write tools that transform or otherwise refactor user-provided source code, such as the one in CL 137076.

CC @matloob @alandonovan @ianthehat @rsc

@gopherbot gopherbot added this to the Unreleased milestone Sep 27, 2018
Copy link

@alandonovan alandonovan commented Sep 27, 2018

The assumption "absolute <=> generated" doesn't hold for other build systems such as Bazel, in which a package may contain a mixture of sources from different directories. Indeed, it doesn't even hold for go list in the presence of modules, since source files may reside in the module cache.

The go/packages API could in principle record which files are generated (and at one point it did), but we removed on the grounds that refactoring tools should decide which files to edit based on the "DO NOT EDIT" comment convention proposed in

Copy link
Member Author

@bcmills bcmills commented Sep 27, 2018

The DO NOT EDIT convention relies on three assumptions:

  1. Generated code is not checked in to the repository.
  2. Users never want refactoring tools to apply even trivial edits to files within the repository that say DO NOT EDIT.
  3. All generated files actually follow the convention.

I don't know about (3), but I do not believe that (1) or (2) holds outside of Blaze/Bazel.

Copy link
Member Author

@bcmills bcmills commented Sep 27, 2018

In particular, you could imagine a workflow that goes:

  1. Refactor the code using some tool, including arbitrary package moves.
  2. Re-run the generators for all generated files that were touched.

Step 1 should not require pruning or excluding checked-in generated files, since they may be needed to type-check packages at any intermediate state.

@bcmills bcmills changed the title x/tools/go/packages: can't distinguish generated sources from user-provided ones x/tools/go/packages: can't distinguish dynamically-generated sources from checked-in ones Sep 28, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants