Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Blob reuse decisions do not take into account manifest support #2205

Closed
mtrmac opened this issue Nov 30, 2023 · 1 comment · Fixed by #2213
Closed

Blob reuse decisions do not take into account manifest support #2205

mtrmac opened this issue Nov 30, 2023 · 1 comment · Fixed by #2213
Labels
kind/bug A defect in an existing functionality (or a PR fixing it)

Comments

@mtrmac
Copy link
Collaborator

mtrmac commented Nov 30, 2023

E.g. when pushing a schema2 image, dockerImageDestination.TryReusingBlobWithOptions can find and choose to use a Zstd layer

  • That directly breaks if the user forced schema1/schema2 manifest type
  • Even without copy.Options.ForceManifestMIMEType, if the destination is manifest-schema-agnostic, and determineManifestConversion has no reason to expect Zstd to be involved, the copy code will only have a single candidate; we need to either add an OCI manifest schema candidate, or not reuse Zstd layers if the copy is not set up to use OCI.
@mtrmac
Copy link
Collaborator Author

mtrmac commented Jan 30, 2024

A reproducer for the first part:

# Create a Zstd-compressed image
skopeo --override-os linux --insecure-policy copy --dest-tls-verify=false --dest-compress-format zstd docker://quay.io/libpod/alpine docker://…/zstd
# The above does not compute/record an uncompressed digest in the BIC, making reuse impossible;
# force decompression to create the uncompressed digest records
skopeo --override-os linux --insecure-policy copy --dest-decompress=true --tls-verify=false docker://quay.io/libpod/alpine dir:t
skopeo --override-os linux --insecure-policy copy --dest-decompress=true --tls-verify=false docker://…/zstd dir:t
# A plain copy now finds the Zstd layer on the destination, and forces a conversion to OCI
skopeo --override-os linux --insecure-policy copy --dest-tls-verify=false docker://quay.io/libpod/alpine docker://…/dest2
# … and a copy forcing manifest schema to non-OCI fails:
skopeo --override-os linux --insecure-policy copy --dest-tls-verify=false --format schema2 docker://quay.io/libpod/alpine docker://…/dest-s2
# … creating an updated image manifest: preparing updated manifest, layer \"sha256:…\": zstd compression is not supported for type \"application/vnd.docker.image.rootfs.diff.tar.gzip\"
# With the fix:
# level=debug msg="Ignoring candidate blob sha256:… in …/dest2, compression zstd does not match required nil or MIME types []string{\"application/vnd.docker.distribution.manifest.v2+json\"}"

The second part is not as easy to reproduce; only dir is manifest-schema-agnostic, but it also does not reuse layers across copies.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug A defect in an existing functionality (or a PR fixing it)
Projects
None yet
1 participant