redundant whiteouts cause a path error

[cjdcordeiro]

Problem

When building an image where new packages are added (either via stage-packages or overlay-packages), the corresponding OCI layer will flag some of the underlying part's files to be whiteout.

Not only this is redundant (because the introduction and deletion of the package files happen within the same part/layer), but it also causes a path error to some tools (like dive).

In short, whiteout should only be used to flag files from underlying layers for deletion, but not files from the same layer.

How to reproduce

rockcraft.yaml:

name: hello
version: "1.0"
base: ubuntu:20.04

parts:
    hello:
        plugin: nil
        stage-packages:
            - hello

Then:

$ rockcraft pack
...
$ skopeo copy oci-archive:hello_1.0.rock docker-daemon:test-image:latest
$ dive test-image:latest
Image Source: docker://test-image:latest
Fetching image... (this can take a while for large images)
Analyzing image...
Building cache...
  path error at layer index Index(0:1): unable to remove '/usr/bin/hello.dpkg-new': path does not exist: /usr/bin/hello.dpkg-new
  path error at layer index Index(0:1): unable to remove '/usr/share/doc/hello.dpkg-new': path does not exist: /usr/share/doc/hello.dpkg-new
  path error at layer index Index(0:1): unable to remove '/usr/share/doc/hello/changelog.Debian.gz.dpkg-new': path does not exist: /usr/share/doc/hello/changelog.Debian.gz.dpkg-new
  path error at layer index Index(0:1): unable to remove '/usr/share/doc/hello/copyright.dpkg-new': path does not exist: /usr/share/doc/hello/copyright.dpkg-new
  path error at layer index Index(0:1): unable to remove '/usr/share/info/hello.info.gz.dpkg-new': path does not exist: /usr/share/info/hello.info.gz.dpkg-new
  path error at layer index Index(0:1): unable to remove '/var/cache/apt/archives/hello_2.10-2ubuntu2_amd64.deb': path does not exist: /var/cache/apt/archives/hello_2.10-2ubuntu2_amd64.deb
  path error at layer index Index(0:1): unable to remove '/var/lib/apt/extended_states.1NJeQR': path does not exist: /var/lib/apt/extended_states.1NJeQR
  path error at layer index Index(0:1): unable to remove '/var/lib/apt/extended_states.3b7RY9': path does not exist: /var/lib/apt/extended_states.3b7RY9
  path error at layer index Index(0:1): unable to remove '/var/lib/dpkg/status-new': path does not exist: /var/lib/dpkg/status-new
  path error at layer index Index(0:1): unable to remove '/var/lib/dpkg/tmp.ci': path does not exist: /var/lib/dpkg/tmp.ci
  path error at layer index Index(0:1): unable to remove '/var/lib/dpkg/info/hello.list-new': path does not exist: /var/lib/dpkg/info/hello.list-new
file tree has path errors

Origin of the problem

The new layer created by the rockcraft part has all of the whiteout files listed in the dive error above. This can be confirmed by:

$ skopeo copy oci-archive:hello_1.0.rock oci:test-image
$ ls -l test-image/blobs/sha256/
total 33824
-rw-r--r-- 1 cris cris      508 Apr 25 10:12 8a023a80af56db32c245ed2fb0a542eb44ceabc24695abcf26a9d9a8095689a3
-rw-r--r-- 1 cris cris  6033508 Apr 25 10:12 8a9350d62b7b6564badccbd007a354a6eab0ddf7aeb1006b5692e179d2e244e8
-rw-r--r-- 1 cris cris 28565998 Apr 25 10:12 8e5c1b329fe39c318c0d49821b339fb94a215c5dc0a2898c8030b5a4d091bcba
-rw-r--r-- 1 cris cris      734 Apr 25 10:12 e41e5ca3f81a4dadba93d0fd71f2ee8c3ef693620933b65b0d325fb490162370
$
$ tar tf test-image/blobs/sha256/8a9350d62b7b6564badccbd007a354a6eab0ddf7aeb1006b5692e179d2e244e8 | grep wh
./var/cache/apt/archives/.wh.hello_2.10-2ubuntu2_amd64.deb
./var/lib/dpkg/.wh.status-new
./var/lib/dpkg/.wh.tmp.ci
./var/lib/dpkg/info/.wh.hello.list-new
./var/lib/apt/.wh.extended_states.3b7RY9
./var/lib/apt/.wh.extended_states.1NJeQR
./usr/bin/.wh.hello.dpkg-new
./usr/share/doc/.wh.hello.dpkg-new
./usr/share/doc/hello/.wh..wh..opq
./usr/share/doc/hello/.wh.changelog.Debian.gz.dpkg-new
./usr/share/doc/hello/.wh.copyright.dpkg-new
./usr/share/info/.wh.hello.info.gz.dpkg-new

These whiteout files do not exist in layer 8e5c1b329fe39c318c0d49821b339fb94a215c5dc0a2898c8030b5a4d091bcba, and thus the whiteout is redundant.

[cjdcordeiro]

@cmatsuoka can this be closed?