c8d/push: Fetch missing resources that can't be mounted #80
Conversation
daemon/containerd/lazy_push.go
Outdated
| fetchHandler := containerdimages.Handlers( | ||
| remotes.FetchHandler(store, fetcher), | ||
| appendDistributionSourceLabel, | ||
| appendLabelHandler(ctx, store, "docker.io/fetch.reason", "push"), |
There was a problem hiding this comment.
I added the label for debugging/testing purposes. I wanted to remove this for PR, but maybe this could be useful, eg. for prune?
We may want to use other label though.
daemon/containerd/lazy_push.go
Outdated
| } | ||
| log = log.WithField("ref", ref.String()) | ||
|
|
||
| name, resolved, err := resolver.Resolve(ctx, ref.String()) |
There was a problem hiding this comment.
This fails currently for blobs that at over 4MiB due to a containerd bug.
I'll open a PR on containerd side to fix this upstream.
Anyway, I wonder if we even need to do the Resolve - looking at the Resolver implementation that we use the name returned is exactly the same as the ref.String(), so only the resolved descriptor is really filled with data from the registry.
However, we already have a descriptor that should have correct digest, mediaType and size, so I think we could drop the Resolve entirely.
Any thoughts on this?
| return registry | ||
| } | ||
|
|
||
| func (source distributionSource) GetReference(dgst digest.Digest) (reference.Named, error) { |
There was a problem hiding this comment.
nit: does this method (and the Registry one) really need to be public?
There was a problem hiding this comment.
Well since the struct itself is private then at this point it doesn't need to be public.
But since those functions are used "outside" of the struct itself then it made sense to me to make them public. It also could be extracted to some other package in future, for example if we'd need to also apply the lazy fetching to other operations.
vvoland
force-pushed
the
c8d-push-lazy-fetch
branch
from
8d94ad5
to
8d392ab
Compare
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
vvoland
force-pushed
the
c8d-push-lazy-fetch
branch
from
8d392ab
to
cfaba5b
Compare
Follow up to:
Lazily fetch content that is needed for push.
Small content like manifests/indexes/configs are always fetched, as they can't be cross-repo mounted and they are small json files.
Other blobs are not fetched if the push registry matches the source registry - in such case they can be cross-repo mounted and there is no need to fetch them.
This makes the pull-tag-push workflow work:
It also works with other registries than Hub (HTTPS only), but in this case all content is fetched from the source registry and then pushed to the target registry.
The progress is currently a bit wonky as it doesn't show the correct status for the lazily fetched content - will fix it in a follow up.
Signed-off-by: Paweł Gronowski pawel.gronowski@docker.com