Warn users when producing an image with files not listed in the SBOM

[luhring]

Context: One of apko's core features is that it produces an SBOM for every image it builds. We want these SBOMs to be as complete as possible.

It's becoming possible for apko's SBOM information to be sourced primarily from individual apks, thanks to Melange.

But it's still possible that apko ends up producing an SBOM that's missing information about files in the image. Files not explicitly documented (e.g. via package metadata) are sometimes referred to as "dark files", since downstream consumers have no visibility into why these files are present.

What's needed: Apko should explicitly warn users when it produces an SBOM that doesn't account for all of the files (regular, directory, etc.) in the image.

[dlorenc]

Could you explain a case where it's possible for apko to generate one of these images? I have a few guesses but I'm not sure I understand.

[luhring]

Sure, I think there are two main cases here:

1. When there's a problem with an apk's SBOM — either the apk didn't come with an SBOM, or it did but the SBOM doesn't include all of the package's files
2. When files are added to the image that don't come from any apk — this can happen with /etc/os-release (in the case highlighted by Warn users when producing an image with an unknown distro #447), or with other files that the apk tool itself uses (e.g. files in /lib/apk/db/...).

In the way I'm thinking about it, there should not be a difference between the outputs of these commands:

All files in the image:

crane export --platform linux/amd64 ${myImage} - | tar -tf - | sed -e 's:^:/:' | sort

All files listed in the SBOM:

cosign download sbom --platform linux/amd64 ${myImage} 2>/dev/null | jq -r '.files | map(.fileName) | sort []'