An API for caching and execution of actions on a remote system.
Branch: master
Clone or download
EricBurnett and ola-rozenfeld Clarify semantics around do_not_cache and skip_cache_lookup (#55)
Clarify semantics for do_not_cache and skip_cache_lookup fields to cover the semantics of when in-flight actions may be "merged", and to clarify when results are and aren't eligible for caching.

The use-cases of interest I'm considering are:
  - Minimizing total number of executions for idempotent actions. (The default: neither field is set; only one total execution is required)
 - Bypassing and overwriting the Action Cache when its results are stale or corrupt. (skip_cache_lookup should be set and the result should still be cached).
 - Actions with side-effects that should always be re-executed (do_not_cache should be set, multiple in-flight requests should not be merged)
 - Actions intentionally being (re)run, e.g. for performance analysis across multiple attempts. (do_not_cache should be set; multiple in-flight requests should not be merged).

I took a look at Bazel and I believe its usage to be consistent with these semantics:
 - --runs_per_test implies uncacheable: https://github.com/bazelbuild/bazel/blob/master/src/main/java/com/google/devtools/build/lib/analysis/test/TestRunnerAction.java#L373
 - Outputs found to be missing after an action cache entry is found result in re-execution with skip_cache_lookup set so the action gets re-executed: https://github.com/bazelbuild/bazel/blob/master/src/main/java/com/google/devtools/build/lib/remote/RemoteSpawnRunner.java#L209
Latest commit cfe8e54 Feb 12, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
build/bazel
.gitignore
AUTHORS Initial commit: licensing and repository info. Mar 28, 2018
BUILD.googleapis Add a quick and dirty build with bazel. Jun 11, 2018
CODEOWNERS
CONTRIBUTING.md Adding another ref Jun 14, 2018
CONTRIBUTORS Initial commit: licensing and repository info. Mar 28, 2018
LICENSE Initial commit: licensing and repository info. Mar 28, 2018
README.md fix typo Nov 16, 2018
WORKSPACE Add a quick and dirty build with bazel. Jun 11, 2018

README.md

Remote Execution API

The Remote Execution API is an API that, at its most general, allows clients to request execution of binaries on a remote system. It is intended primarily for use by build systems, such as Bazel, to distribute build and test actions through a worker pool, and also provide a central cache of build results. This allows builds to execute faster, both by reusing results already built by other clients and by allowing many actions to be executed in parallel, in excess of the resource limits of the machine running the build.

There are a number of clients and services using these APIs, they are listed below.

Clients

These tools use the Remote Execution API to distribute builds to workers.

Servers

These applications implement the Remote Execution API to server build requests from the clients above. These are then distributed to workers; some of these workers implement the Remote Worker API.

Dependencies

The APIs in this repository refer to several general-purpose APIs published by Google in the Google APIs repository. You will need to refer to packages from that repository in order to generate code using this API. If you build the repository using the included BUILD files, Bazel will fetch the protobuf compiler and googleapis automatically.

Using the APIs

The repository contains BUILD files to build the protobuf library with Bazel. If you wish to use them with your own project in Bazel, you will possibly want to declare cc_proto_library, java_proto_library, etc. rules that depend on them.

Other build systems will have to run protoc on the protobuf files, and link in the googleapis and well-known proto types, manually.