-
Notifications
You must be signed in to change notification settings - Fork 4.3k
Closed
Labels
P1I'll work on this now. (Assignee required)I'll work on this now. (Assignee required)team-Remote-ExecIssues and PRs for the Execution (Remote) teamIssues and PRs for the Execution (Remote) teamtype: feature request
Description
This issue tracks the progress towards Remote Builds without the Bytes.
Milestones
- Implement basic support for not downloading intermediate artifacts (In Bazel 0.25)
The feature can be enabled by adding the flag--experimental_remote_download_outputs=minimalto the remote caching / execution invocation. The feature mostly works, but comes with the limitations stated in the subsequent tasks. - Fix local actions that need to fetch remote outputs (In Bazel 0.25)
While Bazel can run most actions remotely there are a few dozen or so action implementations that currently can't. This can be problematic because such actions often require Bazel to download remote files which this change tries to avoid. An example of such an action is theSymlinkAction. - Implement fetching of top level artifacts (In Bazel 0.26)
Implement--experimental_remote_download_outputs=toplevelas described in the design document. - Support downloading unzipped undeclared test outputs
See --remote_download_toplevel --nozip_undeclared_test_outputs does not download any undeclared test outputs for passing tests #17884 for details - Store per output file metadata in the persistent action cache
The persistent action cache stores the state of Bazel's output base. It does not know about remotely stored files. We need to extend it to do so. See Store remote metadata in the on disk action cache #8248 - Builds without the Bytes fails on missing AC result
See Builds without the Bytes fails on missing AC result #10880 for details - Support remote cache eviction
As detailed in the design document it's possible for Bazel to wrongly think that an output file exists on the remote system because it has been evicted ("garbage collected") in the meantime. If a Bazel action then depends on an evicted file there is currently no other way but to fail the build/test. See Remote Builds without the Bytes should support remote output eviction #8250 - Add up- and download stats to the profiler
A user should have insight into what is being up/downloaded and why. - Be compatible with the Build Event Service local file uploader (In Bazel 0.28.0)
This change currently doesn't interact well with the BEP and BES. See Remote Builds without the Bytes should work with BES #8249 - Don't create a runfiles tree for remote actions
For a test that runs remotely we currently create a runfiles tree in the output base with dangling symlinks that's unused. We need to fix theSymlinkTreeStrategy. - Download targets of symlinks in runfiles
See --remote_download_toplevel does not download symlink targets of runfiles #18249 for details - Support symlinks into tree artifacts
See Action input symlink into directory doesn't work with --remote_download_minimal #15678 for details - ctx.actions.symlink actions are not constant time
See FR?: ctx.actions.symlink action time scales w/ size of input; should be constant #14125 for details - Support unresolved symlinks in action outputs
See Symlinks in action outputs are not yet supported by --experimental_remote_download_outputs=minimal #13355 for details - Do not create runfile trees unnecessarily when building without the bytes
See Do not create runfile trees unnecessarily when building without the bytes #18580 for details - Bazel query silently thrashes the analysis cache when --remote_download_minimal is enabled
See Bazel query silently thrashes the analysis cache #10902 for details - Build event file creation fails with symlinks when using --remote_download_minimal
See Build event file creation fails with symlinks when using --remote_download_minimal #11942 for details - Support for mobile-install command
See mobile-install fails with BWOB #14523 for details - Work seamlessly with IDEs
Related Issues
Globegitter, ash2k, vielmetti, adamhowardprice, scele and 37 moreexcitoon, robbertvanginkel and prestonvanloonscele, jin, vmohan3, cocasema, utsav-dbx and 8 moreexcitoon, prestonvanloon and lingyv-lijiridanek
Metadata
Metadata
Assignees
Labels
P1I'll work on this now. (Assignee required)I'll work on this now. (Assignee required)team-Remote-ExecIssues and PRs for the Execution (Remote) teamIssues and PRs for the Execution (Remote) teamtype: feature request