-
Notifications
You must be signed in to change notification settings - Fork 369
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
go_repository: add 'clean' build_file_generation #1802
Conversation
I've marked this PR as draft as I have not yet added tests, but I am planning to. |
This is highly appreciated, let me know if you get stuck anywhere. |
Cc @linzhp for googleapis |
1cf1c77
to
d0d39df
Compare
6adecf6
to
6939d6a
Compare
Ok, I've added tests. Turns out it broke on Windows, so I fixed that and then it broke on Bazel 6.x, so I ended up adding the code into cmd/fetch_repo. My initial attempt was to add it into Gazelle itself, however between fetch_repo and Gazelle the go_repository code first writes a BUILD file with directives, and that would have been removed/ignored by my changes. Tests are green now 😄 This adds a test to see if this error is fixed:
I'm not testing any of the googleapis things I mentioned: since googleapis is technically not a Go repository, getting it working is a happy side-effect. |
6939d6a
to
3329c41
Compare
I've added one more commit on top, reordering (and deduplicating) some code so that fetch_repo is always called. From the commit message:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@tyler-french Could you test this?
Thanks for getting rid of some unnecessary special casing, the code is easier to follow now even though it's longer. |
Before bzlmod, repositories were global, and BUILD files inside upstream repositories could load from the repositories defined at the root-level WORKSPACE file. With the introduction of visibility rules, these result in errors that cannot trivially be worked around. One pattern that has emerged is ignoring existing build files by specifying `gazelle:build_file_name BUILD.bazel`, however this does not work for anything already using BUILD.bazel filenames. This does affect real-world users: an example is github.com/google/go-jsonnet which has a BUILD.bazel file to `genrule` its AST. The module is pure-go, but the build step contains cpp code, and when imported under bzlmod-gazelle will not work because the cpp code is invoked. As a workaround, there is now a 'jsonnet' BCR module. This patch introduces a `build_file_generation = clean` option, which removes existing build files before generating new ones. It allows loading the jsonnet code once again, and also makes it possible to load some of the complex protobuf repositories.
After my changes to remove build files as part of fetch_repo, I realized that when urls=[] is specified the code path doesn't use fetch_repo at all. In this case, fetch_repo_args would remain None, and raise an error. Not ideal! This left me with two general options: either I move the code into a separate command invocation, or I have all code go through fetch_repo. The latter seems to be the pragmatic solution: gazelle by itself doesn't generate code with urls=[] (in either bzlmod or vanilla update-repos), so it's the option with lower performance impact (also considering we're stacking it on top of a network download).
750e401
to
1fc5587
Compare
What type of PR is this?
What package or component does this PR mostly affect?
What does this PR do? Why is it needed?
Before bzlmod, repositories were global, and BUILD files inside upstream repositories could load from the repositories defined at the root-level WORKSPACE file.
With the introduction of visibility rules, these result in errors that cannot trivially be worked around. One pattern that has emerged is ignoring existing build files by specifying
gazelle:build_file_name BUILD.bazel
, however this does not work for anything already using BUILD.bazel filenames.This does affect real-world users: an example is https://github.com/google/go-jsonnet which has a BUILD.bazel file to
genrule
its AST. The module is pure-go, but the build step contains cpp code, and when imported under bzlmod-gazelle will not work because the cpp code is invoked. As a workaround, there is now a 'jsonnet' BCR module.This patch introduces a
build_file_generation = clean
option, which removes existing build files before generating new ones. It allows loading the jsonnet code once again, and also makes it possible to load some of the complex protobuf repositories.Which issues(s) does this PR fix?
Fixes #1773, #1769, #1535, bazelbuild/rules_go#3685
Other notes for review
I have not made 'clean' the default, though I think that for most use cases it will be the correct behavior.
An example for fixing jsonnet:
With some gymnastics this does also make it possible to import
googleapis
as if it were a regular Go dependency, without any switched_rules: