sync with a yaml src should have a "dry-run" or validate mode

[nairb774]

We keep a yaml file checked in for performing syncs, but it doesn't seem like there is a good way to either locally or in some automated way verify that nothing was fat-fingered. For example, if the source yaml looked something like:

gcr.io:
  images:
    distroless/static-debian10:
      - latest
      - nonroot

It would be nice to be able to make sure that the relevant repository[1] exists, and in the case where there are explicit tags listed, it would be nice to make sure that they also exist. Today this is possible by just syncing to a local dir, but that requires going through and actually downloading everything. I was hoping for something a little lighter - possibly only skipping the actual image download.

Would something like this be possible? It might make it possible for us to wire up a CI check that could execute fast, and wouldn't need somewhere to dump tons of data (or need to download it in the first place). Basically anything that would help with improving confidence in editing the yaml file without needing to do the actual sync would be appreciated.

[1] I hope I'm not swapping words. Repository and registry get swapped all the time in my mind.

[vrothberg]

Thanks for reaching out!

To make sure I understood the request correctly: you're looking for a way to check whether the specified sources exist?

I think a --dry-run does make sense. Typos happen quickly and a --dry-run can help.

Cc: @mtrmac @rhatdan @lsm5

[lsm5]

Don't think I fully understand this, but then again, I'm still a n00b to skopeo fwiw. Is this something that could exist in or be useful to skopeo's default CI?

We have an upcoming switch to Cirrus CI coming up sometime soon (yet to begin, no ETA), so maybe we could include this in there as well.

/cc @cevich

[mtrmac]

I think a --dry-run does make sense. Typos happen quickly and a --dry-run can help.

A “compute the full list of images, log that, but don’t copy” mode does seem useful. (For some reason I thought there has already been a PR, but I can’t find one now.)

---

Is this something that could exist in or be useful to skopeo's default CI?

Not trivially — strictly speaking it doubles the test matrix of the sync subcommand :) OTOH some of the existing skopeo sync tests really test the “decide on a set of images to copy” logic (e.g. that the regex option works), and those could use --dry-run to decrease the test cost.

[mtrmac]

One more thing: The trivial implementation of --dry-run would not tell the user that “skopeo sync has tested 100 tags and there’s nothing to update”; it would only list the 100 tags affected. The “there is nothing to update” state is racy anyway, it might not be true by the time the command finishes executing.

[cevich]

@lsm5 in case it helps, I think this is a suggestion for a new skopeo feature to assist with container-centric CI jobs. Not directly with the Travis -> Cirrus conversion, though after it's implemented, it could certainly be a future improvement. Similar to the Ext. Service Check job we run at the beginning of podman's CI.

[mfriedenhagen]

It would be great if the --dry-run could show the number of image versions skopeo did find (or return all matching tags as a list/json array for further processing with jq). Background: we use Artifactory as registry and it allows to specify a maximum number of tags for one image. So we could more easily check beforehand whether a new regex definition finds too many tags and abort the sync.

[github-actions]

A friendly reminder that this issue had no activity for 30 days.

[james-callahan]

We would love to see a --dry-run flag: we would use it as part of our CI flow on pull requests to note which (additional) images need to be mirrored to the reviewer.

[mtrmac]

FWIW this is 99% done in the linked #1459.

[mtrmac]

#1608 was merged.

[jackivanov]

Can this be included in a release?

[mtrmac]

Sure.

@rhatdan is planning what we call a “vendor dance” soon, does that include a Skopeo release? Or should we make one independently?

[jackivanov]

I would be great to see this feature cherry-picked to a release, since the last release was more than a month ago