Skip to content

Commit

Permalink
Move cache instead of copying
Browse files Browse the repository at this point in the history
In `keep-ecdsa` we've seen a number of workflows failing with `No space
lefton device` error.
The problem was a combination of a big cache size and significant memory
allocation after building Docker image.
According to moby/buildkit#1850:
"At the moment caches are copied over the existing cache so it keeps
growing".
As a temporary fix (until issue gets fixed by GH), we introduced a step
that moves the cache (similarily to how it's described in
moby/buildkit#1896) instead of copying.
Although we havent seen the problems with cache size yet in other
projects than `keep-ecdsa`, we're applying the solution across the
repositories to decrease the likelyhood of encountering problem in the
future.
  • Loading branch information
michalinacienciala committed Jun 16, 2021
1 parent eaeffdc commit 251aa89
Show file tree
Hide file tree
Showing 3 changed files with 33 additions and 3 deletions.
12 changes: 11 additions & 1 deletion .github/workflows/client.yml
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ jobs:
# VERSION= ? TODO: Configure version, sample: 1.7.6
load: true # load image to local registry to use it in next steps
cache-from: type=local,src=/tmp/.buildx-cache
cache-to: type=local,dest=/tmp/.buildx-cache
cache-to: type=local,dest=/tmp/.buildx-cache-new

- name: Run Go tests
run: |
Expand Down Expand Up @@ -108,6 +108,16 @@ jobs:
tags: ${{ env.GCR_REGISTRY_URL }}/${{ env.GOOGLE_PROJECT_ID }}/${{ env.IMAGE_NAME }}
labels: revision=${{ github.sha }}
push: ${{ github.event_name == 'workflow_dispatch' }}

- # Temp fix - move cache instead of copying (added below step and
# modified value of `cache-to`).
# https://github.com/docker/build-push-action/issues/252
# https://github.com/moby/buildkit/issues/1896
# Without the change some jobs were failing with `no space left on device`
name: Move cache
run: |
rm -rf /tmp/.buildx-cache
mv /tmp/.buildx-cache-new /tmp/.buildx-cache
- name: Notify CI about completion of the workflow
if: github.event_name == 'workflow_dispatch'
Expand Down
12 changes: 11 additions & 1 deletion .github/workflows/contracts.yml
Original file line number Diff line number Diff line change
Expand Up @@ -254,7 +254,17 @@ jobs:
revision=${{ github.sha }}
push: true
cache-from: type=local,src=/tmp/.buildx-cache
cache-to: type=local,dest=/tmp/.buildx-cache
cache-to: type=local,dest=/tmp/.buildx-cache-new

- # Temp fix - move cache instead of copying (added below step and
# modified value of `cache-to`).
# https://github.com/docker/build-push-action/issues/252
# https://github.com/moby/buildkit/issues/1896
# Without the change some jobs were failing with `no space left on device`
name: Move cache
run: |
rm -rf /tmp/.buildx-cache
mv /tmp/.buildx-cache-new /tmp/.buildx-cache
- name: Notify CI about completion of the workflow
uses: keep-network/notify-workflow-completed@v1
Expand Down
12 changes: 11 additions & 1 deletion .github/workflows/dashboard-testnet.yml
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,17 @@ jobs:
labels: revision=${{ github.sha }}
push: ${{ github.event_name == 'workflow_dispatch' }}
cache-from: type=local,src=/tmp/.buildx-cache
cache-to: type=local,dest=/tmp/.buildx-cache
cache-to: type=local,dest=/tmp/.buildx-cache-new

- # Temp fix - move cache instead of copying (added below step and
# modified value of `cache-to`).
# https://github.com/docker/build-push-action/issues/252
# https://github.com/moby/buildkit/issues/1896
# Without the change some jobs were failing with `no space left on device`
name: Move cache
run: |
rm -rf /tmp/.buildx-cache
mv /tmp/.buildx-cache-new /tmp/.buildx-cache
- name: Notify CI about completion of the workflow
if: github.event_name == 'workflow_dispatch'
Expand Down

0 comments on commit 251aa89

Please sign in to comment.