Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve caching in CI #128

Merged
merged 1 commit into from
Jan 18, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .bazelrc
Original file line number Diff line number Diff line change
Expand Up @@ -35,3 +35,7 @@ coverage --combined_report=lcov
coverage --experimental_generate_llvm_lcov
# Needed because our tests are in a different package than the code they test.
coverage --instrumentation_filter='//.*'

# buildbuddy suggestions. Doesn't hurt to have them always.
build --nolegacy_important_outputs
build --remote_cache_compression
64 changes: 44 additions & 20 deletions .github/workflows/check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,14 @@ jobs:
steps:
- uses: actions/checkout@v3

- name: mount bazel disk cache
uses: actions/cache@v3
with:
path: "~/.cache/bazel_disk_cache"
key: bazel-disk-test-${{ matrix.toolchain }}-${{ matrix.feature }}
- name: set buildbuddy-api-key
run: |
echo 'build --remote_header=x-buildbuddy-api-key=${{ secrets.BUILDBUDDY_API_KEY }}' >> .bazelrc

- name: set bazel upload flags
if: github.ref != 'refs/heads/master'
run: |
echo 'build --noremote_upload_local_results' >> .bazelrc

- name: mount bazel other caches
# restore only on these jobs because GitHub free doesn't
Expand Down Expand Up @@ -57,11 +60,14 @@ jobs:
steps:
- uses: actions/checkout@v3

- name: muont bazel disk cache
uses: actions/cache@v3
with:
path: "~/.cache/bazel_disk_cache"
key: bazel-disk-coverage
- name: set buildbuddy-api-key
run: |
echo 'build --remote_header=x-buildbuddy-api-key=${{ secrets.BUILDBUDDY_API_KEY }}' >> .bazelrc

- name: set bazel upload flags
if: github.ref != 'refs/heads/master'
run: |
echo 'build --noremote_upload_local_results' >> .bazelrc

- name: mount bazel other caches
# This is the only one that writes to the cache, not restore.
Expand All @@ -71,6 +77,18 @@ jobs:
# just taking advantage of the fact that this one is
# not a matrix and so only has one job.
uses: actions/cache@v3
if: github.ref == 'refs/heads/master'
with:
path: |
"~/.cache/bazel_install_base"
"~/.cache/bazel_output_base/external"
"~/.cache/bazel_repository_cache"
"~/.cache/bazelisk"
key: bazel-other

- name: mount bazel other caches
uses: actions/cache/restore@v3
if: github.ref != 'refs/heads/master'
with:
path: |
"~/.cache/bazel_install_base"
Expand Down Expand Up @@ -116,11 +134,14 @@ jobs:
steps:
- uses: actions/checkout@v3

- name: muont bazel disk cache
uses: actions/cache@v3
with:
path: "~/.cache/bazel_disk_cache"
key: bazel-disk-build-${{ matrix.flag }}
- name: set buildbuddy-api-key
run: |
echo 'build --remote_header=x-buildbuddy-api-key=${{ secrets.BUILDBUDDY_API_KEY }}' >> .bazelrc

- name: set bazel upload flags
if: github.ref != 'refs/heads/master'
run: |
echo 'build --noremote_upload_local_results' >> .bazelrc

- name: mount bazel other caches
uses: actions/cache/restore@v3
Expand Down Expand Up @@ -152,11 +173,14 @@ jobs:
steps:
- uses: actions/checkout@v3

- name: muont bazel disk cache
uses: actions/cache@v3
with:
path: "~/.cache/bazel_disk_cache"
key: bazel-disk-buildifier
- name: set buildbuddy-api-key
run: |
echo 'build --remote_header=x-buildbuddy-api-key=${{ secrets.BUILDBUDDY_API_KEY }}' >> .bazelrc

- name: set bazel upload flags
if: github.ref != 'refs/heads/master'
run: |
echo 'build --noremote_upload_local_results' >> .bazelrc

- name: mount bazel other caches
uses: actions/cache/restore@v3
Expand Down
7 changes: 6 additions & 1 deletion .github/workflows/ci.bazelrc
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,12 @@ build --color=yes
build --show_timestamps
build --terminal_columns=120

common --disk_cache=/home/runner/.cache/bazel_disk_cache
# remote cache through buildbuddy
build --bes_results_url=https://app.buildbuddy.io/invocation/
build --bes_backend=grpcs://remote.buildbuddy.io
build --remote_cache=grpcs://remote.buildbuddy.io
build --remote_timeout=3600

common --repository_cache=/home/runner/.cache/bazel_repository_cache

startup --host_jvm_args=-Xmx2500m # avoid blowing up the RAM usage.
Expand Down