Cache miss using COPY --chmod to get equal permissions in image

[chrmarti]

I'm trying to use COPY --chmod to still get a cache hit when the permissions on the host filesystem change. (We want to prebuild images in CI for use on developer machines: devcontainers/cli#153.)

Steps to reproduce on a single Linux machine:

• Create the below Dockerfile and a file.txt next to it:

# syntax=docker/dockerfile:1.4
FROM ubuntu:latest
COPY --chmod=0600 ./file.txt /

• Remove all containers and run docker system prune --all to start fresh.
• Run chmod g-w file.txt && docker buildx build --progress plain .. (A second run shows that all layers are coming from the cache.)
• Run chmod g+w file.txt && docker buildx build --progress plain . (Note the changed group write permission on the host file.)
• COPY shows a cache miss when a cache hit is expect. It looks as if the host's file permission go into the cache checksum instead of the --chmod permissions.

Docker and BuildKit versions:

github.com/docker/buildx v0.9.1 ed00243a0ce2a0aee75311b06e32d33b44729689

Client: Docker Engine - Community
 Version:           20.10.17
 API version:       1.41
 Go version:        go1.17.11
 Git commit:        100c701
 Built:             Mon Jun  6 23:02:46 2022
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.17
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.17.11
  Git commit:       a89b842
  Built:            Mon Jun  6 23:00:51 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.8
  GitCommit:        9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6
 runc:
  Version:          1.1.4
  GitCommit:        v1.1.4-0-g5fd4c4d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

[tonistiigi]

Cache checksums happen before the actual copy with specific parameters is executed. This would require a specific buildkit optimization where copy steps asks for a custom checksum implementation based on its parameters.