image/internal: sha512 support for skopeo copy #475
Conversation
|
✅ A new PR has been created in buildah to vendor these changes: containers/buildah#6507 |
|
Packit jobs failed. @containers/packit-build please check. |
|
/packit rebuild-failed |
|
@mtrmac PTAL |
mtrmac
left a comment
mtrmac
left a comment
There was a problem hiding this comment.
Thanks!
ACK to the image part; the dir: part needs a bit more.
| }() | ||
|
|
||
| digester, stream := putblobdigest.DigestIfCanonicalUnknown(stream, inputInfo) | ||
| digester, stream := putblobdigest.DigestIfUnknown(stream, inputInfo) |
There was a problem hiding this comment.
I think the destination-transport-side digest computation must be a more complex logic, see in the earlier PR about the interaction with cannotModifyManifestReason.
… and dirReference.layerPath discards the algorithm name; that does not generalize for other algorithms, we need to move towards agility where adding an extra algorithm is a ~parameter change and does not require any more changes to the “code proper”; i.e. discarding algorithm names is no longer much of an option.
(We need to keep the existing file names for sha256, to retain compatibility. And… do we define a new value for versionPath?!)
There was a problem hiding this comment.
Updated PutBlob to store blob under provided digest algorithm with the algorithm name prepended (except for Canonical) along with a canonical digest hardlink.
$ /usr/bin/ls
dc518581817f4e75a7dcfd35383e67c3ef85438250c17e10090b5a31ab8f68d4 manifest.json sha512-2ee373e378345b35e7966a106c5c0a40a005a13bfc87695d89c5bb217f969c351e73810cf78d3d841237098731cf76878f05af8f8d28176c316681f9422ff688 version
$ diff dc518581817f4e75a7dcfd35383e67c3ef85438250c17e10090b5a31ab8f68d4 sha512-2ee373e378345b35e7966a106c5c0a40a005a13bfc87695d89c5bb217f969c351e73810cf78d3d841237098731cf76878f05af8f8d28176c316681f9422ff688
$
do we define a new value for
versionPath?!)
Doesn't break existing behaviour but there's new stuff, so maybe we should? I'll defer to you.
| if expectedDigest == "" { | ||
| return fmt.Errorf("expected digest is empty") | ||
| } |
There was a problem hiding this comment.
(I’m not sure we need this, .Validate should reject that already. But it doesn’t hurt … apart from the one extra unit test :) .)
| } | ||
| err := expectedDigest.Validate() | ||
| if err != nil { | ||
| return fmt.Errorf("invalid digest format: %q", expectedDigest) |
There was a problem hiding this comment.
Do we want entirely discard err here?
| } | ||
| computedDigest := digestAlgorithm.FromBytes(blob) | ||
| if computedDigest != expectedDigest { | ||
| return fmt.Errorf("Download config.json digest %s does not match expected %s", computedDigest, expectedDigest) |
There was a problem hiding this comment.
This function does not know that the input is a config.json. (Also see the call-site error messages.)
| computedDigest := digest.FromBytes(blob) | ||
| if computedDigest != m.m.ConfigDescriptor.Digest { | ||
| return nil, fmt.Errorf("Download config.json digest %s does not match expected %s", computedDigest, m.m.ConfigDescriptor.Digest) | ||
| expectedDigest := m.m.ConfigDescriptor.Digest |
There was a problem hiding this comment.
(Non-blocking: I’m not sure we need the single-use variable)
| return nil, fmt.Errorf("Download config.json digest %s does not match expected %s", computedDigest, m.m.ConfigDescriptor.Digest) | ||
| expectedDigest := m.m.ConfigDescriptor.Digest | ||
| if err := validateBlobAgainstDigest(blob, expectedDigest); err != nil { | ||
| return nil, fmt.Errorf("config descriptor validation failed: %w", err) |
There was a problem hiding this comment.
Please keep the two implementations consistent where trivially possible.
| "github.com/opencontainers/go-digest" | ||
| ) | ||
|
|
||
| func validateBlobAgainstDigest(blob []byte, expectedDigest digest.Digest) error { |
There was a problem hiding this comment.
This is critical security code; add unit tests for all code paths, please.
`validateBlobAgainstDigest` verifies that the provided blob matches the exepcted digest. If expected digest itself is invalid or unusable, it rejects the blob. Callers don't need to pre-validate the expected digest. This enables `skopeo copy` to work with sha512-digested images. Co-Authored-By: Claude Code <noreply@anthropic.com> Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
When storing blobs with non-canonical digest algorithms (e.g., sha512): - Store the blob under the provided digest algorithm (e.g., "sha512-abc") - Also compute and store a hard link under the canonical digest (sha256) Co-Authored-By: Claude Code <noreply@anthropic.com> Signed-off-by: Lokesh Mandvekar <lsm5@redhat.com>
|
@mtrmac PTA(nother)L . I think I have addressed all comments so far. Used Claude too, but hopefully no AI slop this time 🤞 . LMK. |
3 participants
See individual commits.
Vendored by containers/skopeo#2747