PullImage ignores contents with same diff id

[cardyok]

Description

Since we have moved content fetching into the unpack process in #3870 to enable trending remote lazy-loading snapshotters like overlaybd/estargz to skip downloading process.

But there are senarios when two layer share the same diff ID but has different digestID.

In this case, after first image is fully downloaded and unpacked, some contents of second image( which shares the same diff id with another layer in the first image) won't be downloaded bc snapshotter thing this layer has already been unpacked (since snapshot ID is chainID calculated from diffID).

After that, imageRef exists, ctr i check will consider second image as incomplete, and when we need the content later when we switch snapshot which needs content to unpack again or use some external client-side tools like nerdctl, this content might be unexpectedly missing.

Steps to reproduce the issue

Describe the results you received and expected

Some discussion about this behavior. Should we consider it as by-design/enhancement(since it optimizes disk space utilization) or bug?

What version of containerd are you using?

containerd containerd.io 1.6.21

Any other relevant information

No response

Show configuration if it is related to CRI plugin.

No response

[cardyok]

will solve this issue with external methods (like in nerdctl). containerd/nerdctl#2295