Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

language/go: introduce static dependency resolution mode #1201

Merged
merged 2 commits into from
Mar 17, 2022

Conversation

blico
Copy link
Member

@blico blico commented Mar 15, 2022

What type of PR is this?

Feature

What package or component does this PR mostly affect?

language/go
repo
go_repository

What does this PR do? Why is it needed?

Creates a new dependencyMode called static, which avoids going out to the network when resolving imports to external labels and instead solely relies on the dependencies Gazelle knows about within WORKSPACE. If no matching dependency is found, Gazelle will skip generating a label for it.

This can greatly improve the performance of go_repository rules by eliminating unnecessary network calls when an external dependency contains importpaths that are not used within the main repository (and therefore not needed).

static is now the default dependencyMode for go_repository rules, given that all known dependencies can be provided via WORKSPACE, and any needed unknown dependencies should already result in build failure.

Which issues(s) does this PR fix?

Fixes #602

Other notes for review
A rough experiment shows in static mode it took a go_repository rule for github.com/docker/docker ~16.5s to fetch, compared to ~633 seconds in external mode:

$ vi WORKSPACE # invalidate rule
$ time bazel fetch @com_github_docker_docker_external//...
Loading: 849 packages loaded

real	10m33.033s
user	0m0.621s
sys	0m0.320s
$ vi WORKSPACE # invalidate rule
$ time bazel fetch @com_github_docker_docker_static//...
Loading: 849 packages loaded

real	0m16.587s
user	0m0.499s
sys	0m0.233s

repo/remote.go Show resolved Hide resolved
repo/remote.go Outdated Show resolved Hide resolved
@linzhp linzhp merged commit 9c47513 into bazelbuild:master Mar 17, 2022
l46kok added a commit to l46kok/cel-go that referenced this pull request Jan 25, 2023
See bazelbuild/bazel-gazelle#1201

results:

```
 % time bazel build ...
 INFO: Analyzed 49 targets (0 packages loaded, 0 targets configured).
 INFO: Found 49 targets...
 INFO: Elapsed time: 20.776s, Critical Path: 20.33s
 INFO: 138 processes: 3 internal, 135 linux-sandbox.
 INFO: Build completed successfully, 138 total actions
 bazel build ...  0.01s user 0.01s system 0% cpu 20.797 total
```
l46kok added a commit to l46kok/cel-go that referenced this pull request Jan 25, 2023
See bazelbuild/bazel-gazelle#1201

results:

```
 % time bazel build ...
 INFO: Analyzed 49 targets (0 packages loaded, 0 targets configured).
 INFO: Found 49 targets...
 INFO: Elapsed time: 20.776s, Critical Path: 20.33s
 INFO: 138 processes: 3 internal, 135 linux-sandbox.
 INFO: Build completed successfully, 138 total actions
 bazel build ...  0.01s user 0.01s system 0% cpu 20.797 total
```
TristonianJones pushed a commit to google/cel-go that referenced this pull request Jan 25, 2023
See bazelbuild/bazel-gazelle#1201

results:

```
 % time bazel build ...
 INFO: Analyzed 49 targets (0 packages loaded, 0 targets configured).
 INFO: Found 49 targets...
 INFO: Elapsed time: 20.776s, Critical Path: 20.33s
 INFO: 138 processes: 3 internal, 135 linux-sandbox.
 INFO: Build completed successfully, 138 total actions
 bazel build ...  0.01s user 0.01s system 0% cpu 20.797 total
```
malt3 added a commit to malt3/bazeldnf that referenced this pull request Feb 2, 2023
Consult the following resources for more information:

- bazelbuild/bazel-gazelle#1217
- bazelbuild/bazel-gazelle#1264
- bazelbuild/bazel-gazelle#1201

TL;DR: When using recent versions of gazelle, the newer dependency resolution mode "static" is used by default. This especially fails for golang.org/x/... and when using a macro (_maybe) for the go_repository rules [1]. To avoid the issue, set build mode to external for now.

[1] bazelbuild/bazel-gazelle#1217 (comment)
malt3 added a commit to malt3/bazeldnf that referenced this pull request Feb 2, 2023
Consult the following resources for more information:

- bazelbuild/bazel-gazelle#1217
- bazelbuild/bazel-gazelle#1264
- bazelbuild/bazel-gazelle#1201

TL;DR: When using recent versions of gazelle, the newer dependency resolution mode "static" is used by default. This especially fails for golang.org/x/... and when using a macro (_maybe) for the go_repository rules [1]. To avoid the issue, set build mode to external for now.

[1] bazelbuild/bazel-gazelle#1217 (comment)
malt3 added a commit to malt3/bazeldnf that referenced this pull request Feb 2, 2023
Consult the following resources for more information:

- bazelbuild/bazel-gazelle#1217
- bazelbuild/bazel-gazelle#1264
- bazelbuild/bazel-gazelle#1201

TL;DR: When using recent versions of gazelle, the newer dependency resolution mode "static" is used by default. This especially fails for golang.org/x/... and when using a macro (_maybe) for the go_repository rules [1]. To avoid the issue, set build mode to external for now.

[1] bazelbuild/bazel-gazelle#1217 (comment)
malt3 added a commit to malt3/bazeldnf that referenced this pull request Feb 2, 2023
Consult the following resources for more information:

- bazelbuild/bazel-gazelle#1217
- bazelbuild/bazel-gazelle#1264
- bazelbuild/bazel-gazelle#1201

TL;DR: When using recent versions of gazelle, the newer dependency resolution mode "static" is used by default. This especially fails for golang.org/x/... and when using a macro (_maybe) for the go_repository rules [1]. To avoid the issue, set build mode to external for now.

[1] bazelbuild/bazel-gazelle#1217 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

go_repository: statically resolve external imports
2 participants