OCPBUGS-33196: skip images with both tag and digest #853

lmzuccarelli · 2024-05-14T13:11:53Z

Description

This addresses the issue when images being mirrored, having both tag and digest would cause the mirror execution to fail, the fix ensures these type images will be skipped with an appropriate warning message.

Fixes # OCPBUGS-33196

Type of change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

How Has This Been Tested?

I used the following imagesetconfig for testing

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v2alpha1
mirror:
   additionalImages:
   - name: quay.io/cilium/cilium-etcd-operator:v2.0.7@sha256:04b8327f7f992693c2cb483b999041ed8f92efc8e14f2a5f3ab95574a65ea2dc
   - name: quay.io/coreos/etcd:v3.5.4@sha256:a67fb152d4c53223e96e818420c37f11d05c2d92cf62c05ca5604066c37295e9

Executed the following mirror to disk workflow

bin/oc-mirror --config ocpbugs-33196.yaml file://taganddigest --v2

Expected Outcome

2024/05/14 14:18:35  [WARN]   : ⚠️  --v2 flag identified, flow redirected to the oc-mirror v2 version. This is Tech Preview, it is still under development and it is not production ready.
2024/05/14 14:18:35  [INFO]   : 👋 Hello, welcome to oc-mirror
2024/05/14 14:18:35  [INFO]   : ⚙️  setting up the environment for you...
2024/05/14 14:18:35  [INFO]   : 🔀 workflow mode: mirrorToDisk 
2024/05/14 14:18:35  [INFO]   : 🕵️  going to discover the necessary images...
2024/05/14 14:18:35  [INFO]   : 🔍 collecting release images...
2024/05/14 14:18:35  [INFO]   : 🔍 collecting operator images...
2024/05/14 14:18:35  [INFO]   : 🔍 collecting additional images...
2024/05/14 14:18:35  [WARN]   : [AdditionalImagesCollector] mirroring skipped : source image quay.io/cilium/cilium-etcd-operator:v2.0.7@sha256:04b8327f7f992693c2cb483b999041ed8f92efc8e14f2a5f3ab95574a65ea2dc has both tag and digest
2024/05/14 14:18:35  [WARN]   : [AdditionalImagesCollector] mirroring skipped : source image quay.io/coreos/etcd:v3.5.4@sha256:a67fb152d4c53223e96e818420c37f11d05c2d92cf62c05ca5604066c37295e9 has both tag and digest
2024/05/14 14:18:35  [INFO]   : 🚀 Start copying the images...
2024/05/14 14:18:35  [INFO]   : === Results ===
2024/05/14 14:18:35  [INFO]   : All release images mirrored successfully 0 / 0 ✅
2024/05/14 14:18:35  [INFO]   : All operator images mirrored successfully 0 / 0 ✅
2024/05/14 14:18:35  [INFO]   : All additional images mirrored successfully 0 / 0 ✅
2024/05/14 14:18:35  [INFO]   : 📦 Preparing the tarball archive...
2024/05/14 14:18:35  [INFO]   : mirror time     : 1.770588ms
2024/05/14 14:18:35  [INFO]   : 👋 Goodbye, thank you for using oc-mirror

openshift-ci-robot · 2024-05-14T13:11:59Z

@lmzuccarelli: This pull request references Jira Issue OCPBUGS-33196, which is invalid:

expected the bug to target the "4.16.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

Description

This addresses the issue when images being mirrored, having both tag and digest would cause the mirror execution to fail, the fix ensures these type images will be skipped with an appropriate warning message.

Fixes # OCPBUGS-33196

Type of change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

How Has This Been Tested?

I used the following imagesetconfig for testing

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v2alpha1
mirror:
  additionalImages:
  - name: quay.io/cilium/cilium-etcd-operator:v2.0.7@sha256:04b8327f7f992693c2cb483b999041ed8f92efc8e14f2a5f3ab95574a65ea2dc
  - name: quay.io/coreos/etcd:v3.5.4@sha256:a67fb152d4c53223e96e818420c37f11d05c2d92cf62c05ca5604066c37295e9

Executed the following mirror to disk workflow

bin/oc-mirror --config ocpbugs-33196.yaml file://taganddigest --v2

Expected Outcome

2024/05/14 14:18:35  [WARN]   : ⚠️  --v2 flag identified, flow redirected to the oc-mirror v2 version. This is Tech Preview, it is still under development and it is not production ready.
2024/05/14 14:18:35  [INFO]   : 👋 Hello, welcome to oc-mirror
2024/05/14 14:18:35  [INFO]   : ⚙️  setting up the environment for you...
2024/05/14 14:18:35  [INFO]   : 🔀 workflow mode: mirrorToDisk 
2024/05/14 14:18:35  [INFO]   : 🕵️  going to discover the necessary images...
2024/05/14 14:18:35  [INFO]   : 🔍 collecting release images...
2024/05/14 14:18:35  [INFO]   : 🔍 collecting operator images...
2024/05/14 14:18:35  [INFO]   : 🔍 collecting additional images...
2024/05/14 14:18:35  [WARN]   : [AdditionalImagesCollector] mirroring skipped : source image quay.io/cilium/cilium-etcd-operator:v2.0.7@sha256:04b8327f7f992693c2cb483b999041ed8f92efc8e14f2a5f3ab95574a65ea2dc has both tag and digest
2024/05/14 14:18:35  [WARN]   : [AdditionalImagesCollector] mirroring skipped : source image quay.io/coreos/etcd:v3.5.4@sha256:a67fb152d4c53223e96e818420c37f11d05c2d92cf62c05ca5604066c37295e9 has both tag and digest
2024/05/14 14:18:35  [INFO]   : 🚀 Start copying the images...
2024/05/14 14:18:35  [INFO]   : === Results ===
2024/05/14 14:18:35  [INFO]   : All release images mirrored successfully 0 / 0 ✅
2024/05/14 14:18:35  [INFO]   : All operator images mirrored successfully 0 / 0 ✅
2024/05/14 14:18:35  [INFO]   : All additional images mirrored successfully 0 / 0 ✅
2024/05/14 14:18:35  [INFO]   : 📦 Preparing the tarball archive...
2024/05/14 14:18:35  [INFO]   : mirror time     : 1.770588ms
2024/05/14 14:18:35  [INFO]   : 👋 Goodbye, thank you for using oc-mirror

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

lmzuccarelli · 2024-05-14T13:13:48Z

/jira refresh

openshift-ci-robot · 2024-05-14T13:13:55Z

@lmzuccarelli: This pull request references Jira Issue OCPBUGS-33196, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target version (4.16.0) matches configured target version for branch (4.16.0)
bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @zhouying7780

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci · 2024-05-14T13:28:01Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: lmzuccarelli

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [lmzuccarelli]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

kasturinarra · 2024-05-15T16:00:37Z

Tested, have few questions, waiting for reply from Luigi.

lmzuccarelli · 2024-05-16T08:04:58Z

@kasturinarra - I updated the console message as requested

kasturinarra · 2024-05-16T09:58:41Z

@kasturinarra - I updated the console message as requested

Thank you !! Let me test again

kasturinarra · 2024-05-16T10:29:36Z

@kasturinarra - I updated the console message as requested

Thank you !! Let me test again

could see log has been adjusted. see below

2024/05/16 10:28:44 [WARN] : [AdditionalImagesCollector] quay.io/cilium/cilium-etcd-operator:v2.0.7@sha256:04b8327f7f992693c2cb483b999041ed8f92efc8e14f2a5f3ab95574a65ea2dc has both tag and digest : SKIPPING
2024/05/16 10:28:44 [WARN] : [AdditionalImagesCollector] quay.io/coreos/etcd:v3.5.4@sha256:a67fb152d4c53223e96e818420c37f11d05c2d92cf62c05ca5604066c37295e9 has both tag and digest : SKIPPING

kasturinarra · 2024-05-16T10:29:43Z

/label qe-approved

openshift-ci-robot · 2024-05-16T10:29:49Z

@lmzuccarelli: This pull request references Jira Issue OCPBUGS-33196, which is valid.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target version (4.16.0) matches configured target version for branch (4.16.0)
bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @kasturinarra

In response to this:

Description

This addresses the issue when images being mirrored, having both tag and digest would cause the mirror execution to fail, the fix ensures these type images will be skipped with an appropriate warning message.

Fixes # OCPBUGS-33196

Type of change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

How Has This Been Tested?

I used the following imagesetconfig for testing

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v2alpha1
mirror:
  additionalImages:
  - name: quay.io/cilium/cilium-etcd-operator:v2.0.7@sha256:04b8327f7f992693c2cb483b999041ed8f92efc8e14f2a5f3ab95574a65ea2dc
  - name: quay.io/coreos/etcd:v3.5.4@sha256:a67fb152d4c53223e96e818420c37f11d05c2d92cf62c05ca5604066c37295e9

Executed the following mirror to disk workflow

bin/oc-mirror --config ocpbugs-33196.yaml file://taganddigest --v2

Expected Outcome

2024/05/14 14:18:35  [WARN]   : ⚠️  --v2 flag identified, flow redirected to the oc-mirror v2 version. This is Tech Preview, it is still under development and it is not production ready.
2024/05/14 14:18:35  [INFO]   : 👋 Hello, welcome to oc-mirror
2024/05/14 14:18:35  [INFO]   : ⚙️  setting up the environment for you...
2024/05/14 14:18:35  [INFO]   : 🔀 workflow mode: mirrorToDisk 
2024/05/14 14:18:35  [INFO]   : 🕵️  going to discover the necessary images...
2024/05/14 14:18:35  [INFO]   : 🔍 collecting release images...
2024/05/14 14:18:35  [INFO]   : 🔍 collecting operator images...
2024/05/14 14:18:35  [INFO]   : 🔍 collecting additional images...
2024/05/14 14:18:35  [WARN]   : [AdditionalImagesCollector] mirroring skipped : source image quay.io/cilium/cilium-etcd-operator:v2.0.7@sha256:04b8327f7f992693c2cb483b999041ed8f92efc8e14f2a5f3ab95574a65ea2dc has both tag and digest
2024/05/14 14:18:35  [WARN]   : [AdditionalImagesCollector] mirroring skipped : source image quay.io/coreos/etcd:v3.5.4@sha256:a67fb152d4c53223e96e818420c37f11d05c2d92cf62c05ca5604066c37295e9 has both tag and digest
2024/05/14 14:18:35  [INFO]   : 🚀 Start copying the images...
2024/05/14 14:18:35  [INFO]   : === Results ===
2024/05/14 14:18:35  [INFO]   : All release images mirrored successfully 0 / 0 ✅
2024/05/14 14:18:35  [INFO]   : All operator images mirrored successfully 0 / 0 ✅
2024/05/14 14:18:35  [INFO]   : All additional images mirrored successfully 0 / 0 ✅
2024/05/14 14:18:35  [INFO]   : 📦 Preparing the tarball archive...
2024/05/14 14:18:35  [INFO]   : mirror time     : 1.770588ms
2024/05/14 14:18:35  [INFO]   : 👋 Goodbye, thank you for using oc-mirror

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

sherine-k · 2024-05-16T12:04:19Z

lgtm
I was wondering if there would be a benefit to add these images to the list of images that fail to mirror by batch ?

sherine-k · 2024-05-16T14:04:57Z

/lgtm
/label acknowledge-critical-fixes-only

openshift-ci · 2024-05-16T14:32:57Z

@lmzuccarelli: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

openshift-ci-robot · 2024-05-16T14:37:31Z

@lmzuccarelli: Jira Issue OCPBUGS-33196: All pull requests linked via external trackers have merged:

openshift/oc-mirror#853

Jira Issue OCPBUGS-33196 has been moved to the MODIFIED state.

In response to this:

Description

This addresses the issue when images being mirrored, having both tag and digest would cause the mirror execution to fail, the fix ensures these type images will be skipped with an appropriate warning message.

Fixes # OCPBUGS-33196

Type of change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

How Has This Been Tested?

I used the following imagesetconfig for testing

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v2alpha1
mirror:
  additionalImages:
  - name: quay.io/cilium/cilium-etcd-operator:v2.0.7@sha256:04b8327f7f992693c2cb483b999041ed8f92efc8e14f2a5f3ab95574a65ea2dc
  - name: quay.io/coreos/etcd:v3.5.4@sha256:a67fb152d4c53223e96e818420c37f11d05c2d92cf62c05ca5604066c37295e9

Executed the following mirror to disk workflow

bin/oc-mirror --config ocpbugs-33196.yaml file://taganddigest --v2

Expected Outcome

2024/05/14 14:18:35  [WARN]   : ⚠️  --v2 flag identified, flow redirected to the oc-mirror v2 version. This is Tech Preview, it is still under development and it is not production ready.
2024/05/14 14:18:35  [INFO]   : 👋 Hello, welcome to oc-mirror
2024/05/14 14:18:35  [INFO]   : ⚙️  setting up the environment for you...
2024/05/14 14:18:35  [INFO]   : 🔀 workflow mode: mirrorToDisk 
2024/05/14 14:18:35  [INFO]   : 🕵️  going to discover the necessary images...
2024/05/14 14:18:35  [INFO]   : 🔍 collecting release images...
2024/05/14 14:18:35  [INFO]   : 🔍 collecting operator images...
2024/05/14 14:18:35  [INFO]   : 🔍 collecting additional images...
2024/05/14 14:18:35  [WARN]   : [AdditionalImagesCollector] mirroring skipped : source image quay.io/cilium/cilium-etcd-operator:v2.0.7@sha256:04b8327f7f992693c2cb483b999041ed8f92efc8e14f2a5f3ab95574a65ea2dc has both tag and digest
2024/05/14 14:18:35  [WARN]   : [AdditionalImagesCollector] mirroring skipped : source image quay.io/coreos/etcd:v3.5.4@sha256:a67fb152d4c53223e96e818420c37f11d05c2d92cf62c05ca5604066c37295e9 has both tag and digest
2024/05/14 14:18:35  [INFO]   : 🚀 Start copying the images...
2024/05/14 14:18:35  [INFO]   : === Results ===
2024/05/14 14:18:35  [INFO]   : All release images mirrored successfully 0 / 0 ✅
2024/05/14 14:18:35  [INFO]   : All operator images mirrored successfully 0 / 0 ✅
2024/05/14 14:18:35  [INFO]   : All additional images mirrored successfully 0 / 0 ✅
2024/05/14 14:18:35  [INFO]   : 📦 Preparing the tarball archive...
2024/05/14 14:18:35  [INFO]   : mirror time     : 1.770588ms
2024/05/14 14:18:35  [INFO]   : 👋 Goodbye, thank you for using oc-mirror

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-bot · 2024-05-16T20:54:14Z

[ART PR BUILD NOTIFIER]

This PR has been included in build oc-mirror-plugin-container-v4.16.0-202405161711.p0.g2adde69.assembly.stream.el9 for distgit oc-mirror-plugin.
All builds following this will include this PR.

openshift-merge-robot · 2024-05-18T00:21:58Z

Fix included in accepted release 4.16.0-0.nightly-2024-05-17-180525

openshift-ci-robot added jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. and removed jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels May 14, 2024

openshift-ci bot requested review from zhouying7780, aguidirh and sherine-k May 14, 2024 13:14

lmzuccarelli force-pushed the OCPBUGS-33196 branch from bc5f2d7 to cf93c7b Compare May 14, 2024 13:28

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 14, 2024

lmzuccarelli force-pushed the OCPBUGS-33196 branch from cf93c7b to ebf19a6 Compare May 14, 2024 13:32

OCPBUGS-33196: skip images with both tag and digest

7507908

lmzuccarelli force-pushed the OCPBUGS-33196 branch from ebf19a6 to 7507908 Compare May 16, 2024 08:04

openshift-ci bot added the qe-approved Signifies that QE has signed off on this PR label May 16, 2024

openshift-ci bot requested a review from kasturinarra May 16, 2024 10:29

openshift-ci bot added the acknowledge-critical-fixes-only Indicates if the issuer of the label is OK with the policy. label May 16, 2024

openshift-ci bot assigned sherine-k May 16, 2024

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label May 16, 2024

openshift-merge-bot bot merged commit 2adde69 into openshift:main May 16, 2024
5 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OCPBUGS-33196: skip images with both tag and digest #853

OCPBUGS-33196: skip images with both tag and digest #853

lmzuccarelli commented May 14, 2024

openshift-ci-robot commented May 14, 2024

Description

Type of change

How Has This Been Tested?

Expected Outcome

lmzuccarelli commented May 14, 2024

openshift-ci-robot commented May 14, 2024

openshift-ci bot commented May 14, 2024

kasturinarra commented May 15, 2024

lmzuccarelli commented May 16, 2024 •

edited

Loading

kasturinarra commented May 16, 2024

kasturinarra commented May 16, 2024

kasturinarra commented May 16, 2024

openshift-ci-robot commented May 16, 2024

Description

Type of change

How Has This Been Tested?

Expected Outcome

sherine-k commented May 16, 2024 •

edited

Loading

sherine-k commented May 16, 2024

openshift-ci bot commented May 16, 2024

openshift-ci-robot commented May 16, 2024

Description

Type of change

How Has This Been Tested?

Expected Outcome

openshift-bot commented May 16, 2024

openshift-merge-robot commented May 18, 2024

OCPBUGS-33196: skip images with both tag and digest #853

OCPBUGS-33196: skip images with both tag and digest #853

Conversation

lmzuccarelli commented May 14, 2024

Description

Type of change

How Has This Been Tested?

Expected Outcome

openshift-ci-robot commented May 14, 2024

Description

Type of change

How Has This Been Tested?

Expected Outcome

lmzuccarelli commented May 14, 2024

openshift-ci-robot commented May 14, 2024

openshift-ci bot commented May 14, 2024

kasturinarra commented May 15, 2024

lmzuccarelli commented May 16, 2024 • edited Loading

kasturinarra commented May 16, 2024

kasturinarra commented May 16, 2024

kasturinarra commented May 16, 2024

openshift-ci-robot commented May 16, 2024

Description

Type of change

How Has This Been Tested?

Expected Outcome

sherine-k commented May 16, 2024 • edited Loading

sherine-k commented May 16, 2024

openshift-ci bot commented May 16, 2024

openshift-ci-robot commented May 16, 2024

Description

Type of change

How Has This Been Tested?

Expected Outcome

openshift-bot commented May 16, 2024

openshift-merge-robot commented May 18, 2024

lmzuccarelli commented May 16, 2024 •

edited

Loading

sherine-k commented May 16, 2024 •

edited

Loading