skopeo copy does not preserve digests for certain images #1451
Comments
|
Thanks for your report. The Fundamentally, it’s a very reasonable desire that Skopeo should be able to copy an image without changing its digest at all. Currently that happens when copying to digest references, or when the image is signed; #1378 tracks allowing users to explicitly prohibit image changes, in situations like these. As for workarounds/process changes with the software as is: During uploads, Skopeo primarily tries to avoid the upload; so if the registry is known to contain a version of the blob, it is reused. If uploading, Skopeo compresses files when uploading to registries; that’s typically quite useful when the source uses an uncompressed format, but that policy is independent of the source, and currently there isn’t an opt-out. Creating the image: It might be possible to change how the image was originally created. Either the source image was somehow ~intentionally built with an uncompressed layer and uploaded to a registry that way (which might be possible to fix in the tool that did that); Or maybe Buildah was used to create the final image, and Buildah noticed that the uncompressed version of that layer exists on the registry, so it decided to reuse it instead of uploading the a3e… version; that could be worked around by somehow removing the Copying the image: Alternatively, when taking the image as given, and just wanting to copy it, this might work: (because copies to a digested reference prohibit changing the image, and a later copy in the same location will notice that all necessary objects are already present, and thus not cause a new copy that could possibly compress the data.) |
|
Thanks for the detailed response, this really helps. I tried the suggested workaround for copy and it works. I guess we could have prevented this by using skopeo to push to our first destination icr.io (which we want to be source of truth), since skopeo would have compressed it for us by default during that copy. fwiw, just found that oc image mirror seems to handle this particular scenario. since there's an existing #1378 to address this, pls close this if not needed. |
|
Thanks for the confirmation, let’s track this in #1378 . |
A digest-stable copy seems popular, even when not copying signed images. Using --all can still change digests. Adding an option to ensure digests are preserved. See: containers/skopeo#1440 containers/skopeo#1378 containers/skopeo#1102 containers/skopeo#1451 Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
A digest-stable copy seems popular, even when not copying signed images. Using --all can still change digests. Adding an option to ensure digests are preserved. See: containers/skopeo#1440 containers/skopeo#1378 containers/skopeo#1102 containers/skopeo#1451 Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
A digest-stable copy seems popular, even when not copying signed images. Using --all can still change digests. Adding an option to ensure digests are preserved. See: containers/skopeo#1440 containers/skopeo#1378 containers/skopeo#1102 containers/skopeo#1451 Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
A digest-stable copy seems popular, even when not copying signed images. Using --all can still change digests. Adding an option to ensure digests are preserved. Also adding a missing check to enable digest preservation for manifest lists where the destination is digested. See: containers/skopeo#1440 containers/skopeo#1378 containers/skopeo#1102 containers/skopeo#1451 Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
A digest-stable copy seems popular, even when not copying signed images. Using --all can still change digests. Adding an option to ensure digests are preserved. Also adding a missing check to enable digest preservation for manifest lists where the destination is digested. See: containers/skopeo#1440 containers/skopeo#1378 containers/skopeo#1102 containers/skopeo#1451 Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
A digest-stable copy seems popular, even when not copying signed images. Using --all can still change digests. Adding an option to ensure digests are preserved. Also adding a missing check to enable digest preservation for manifest lists where the destination is digested. See: containers/skopeo#1440 containers/skopeo#1378 containers/skopeo#1102 containers/skopeo#1451 Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
A digest-stable copy seems popular, even when not copying signed images. Using --all can still change digests. Adding an option to ensure digests are preserved. Also adding a missing check to enable digest preservation for manifest lists where the destination is digested. See: containers/skopeo#1440 containers/skopeo#1378 containers/skopeo#1102 containers/skopeo#1451 Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
A digest-stable copy seems popular, even when not copying signed images. Using --all can still change digests. Adding an option to ensure digests are preserved. Also adding a missing check to enable digest preservation for manifest lists where the destination is digested. See: containers/skopeo#1440 containers/skopeo#1378 containers/skopeo#1102 containers/skopeo#1451 Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
A digest-stable copy seems popular, even when not copying signed images. Using --all can still change digests. Adding an option to ensure digests are preserved. Also adding a missing check to enable digest preservation for manifest lists where the destination is digested. See: containers/skopeo#1440 containers/skopeo#1378 containers/skopeo#1102 containers/skopeo#1451 Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
A digest-stable copy seems popular, even when not copying signed images. Using --all can still change digests. Adding an option to ensure digests are preserved. Also adding a missing check to enable digest preservation for manifest lists where the destination is digested. See: containers/skopeo#1440 containers/skopeo#1378 containers/skopeo#1102 containers/skopeo#1451 Signed-off-by: James Hewitt <james.hewitt@uk.ibm.com>
When we copy images between registries using skopeo, we observe that digests are stable for most cases. But occasionally the digests changes at the destination.
Can some explain why this changes or what can we do to prevent this change & preserve the digest if possible ? ( should we build & push image a particular way or use any skopeo flag)
How to reproduce ?
1. Create a temporary registry
2. Copy the image
3. Difference in digests btw source & dest registry
Additional Info:
There seems difference between one image layers between the source and after copy
Related:
The text was updated successfully, but these errors were encountered: