Pretty big cache

[MarcelCoding]

Description

The cache can grow very quickly with large images, since old entries are not deleted.

Configuration

• Repository URL (if public): https://github.com/MarcelCoding/docker-desktop-vnc/blob/02fb4a97d5a75e40c467b856538efa443d1077cc/.github/workflows/docker-publish.yml#L28-L60
• Build URL (if public): https://github.com/MarcelCoding/docker-desktop-vnc/runs/1549446282?check_suite_focus=true#logs

[...]

      - name: Cache Docker layers
        uses: actions/cache@v2
        with:
          path: /tmp/.buildx-cache
          key: ${{ runner.os }}-buildx-${{ github.sha }}
          restore-keys: |
            ${{ runner.os }}-buildx-

[...]

      - name: Build
        uses: docker/build-push-action@v2
        with:
          push: false
          tags: ${{ steps.prepare.outputs.image }}
          platforms: ${{ env.DOCKER_PLATFORMS }}
          cache-from: type=local,src=/tmp/.buildx-cache
          cache-to: type=local,dest=/tmp/.buildx-cache
          context: .

[...]

Logs

logs_37.zip

My solution

Add a clean-cache configuration option that runs the following command before exporting the layers docker system prune -f --filter "until=5h".

[crazy-max]

@MarcelCoding

The cache can grow very quickly with large images, since old entries are not deleted.

Yes you're right atm caches are copied over the existing cache so it keeps growing. Can you open an issue on buildkit repo about that please? In the meantime you can do this:

[...]

      - name: Cache Docker layers
        uses: actions/cache@v2
        with:
          path: /tmp/.buildx-cache
          key: ${{ runner.os }}-buildx-${{ github.sha }}
          restore-keys: |
            ${{ runner.os }}-buildx-

[...]

      - name: Build
        uses: docker/build-push-action@v2
        with:
          push: false
          tags: ${{ steps.prepare.outputs.image }}
          platforms: ${{ env.DOCKER_PLATFORMS }}
          cache-from: type=local,src=/tmp/.buildx-cache
          cache-to: type=local,dest=/tmp/.buildx-cache-new
          context: .

[...]

      - name: Move cache
        run:
          rm -rf /tmp/.buildx-cache
          mv /tmp/.buildx-cache-new /tmp/.buildx-cache

cc. @tonistiigi

[MarcelCoding]

@crazy-max that means that the cache-to option does not export the entire Docker cache, just the one for the current images?

[crazy-max]

@MarcelCoding

that means that the cache-to option does not export the entire Docker cache, just the one for the current images?

cache-to exports the build cache for the current image being built with buildkit yes. If you are implying that the cache can be shared between different images, this is not the case. See #153 (comment).

[MarcelCoding]

Ok, thanks for the help, I will create an issue in the builtkit repo. For implementing an option to clean the cache/remove old versions of the cache.

[zero88]

Hi @crazy-max
Can I use buildkit registry cache within local registry by GitHub service container, then caching registry volume as GitHub cache.
After each run, just prune cache in local registry to keep as small as possible.

However, I'm not sure about GitHub service container is able to create volume for service container and your GitHub action support that (because I'm only see caching inline option)

[malobre]

docker/buildx#535 should fix this and make using github cache a breeze:

[...]

      - name: Build
        uses: docker/build-push-action@v2
        with:
          tags: user/app:latest
          cache-from: type=gha
          cache-to: type=gha

[zero88]

I used it for caching
https://github.com/zero88/gh-registry/blob/main/README.md#usage