OCPBUGS-16801: fix: fixes the bug on catalog when using field archiveSize: 1

[aguidirh]

Description

Currently when the field archiveSize on imageSetConfig is set to 1 the mirroring fails for catalogs.

It was happening because the code was not looping in all archives generated to extract all the catalog content. Also, the folder structure was being kept only in the first archive generated.

The fix is to loop in all archives and also to get all files related with the catalogs/ folder, even if the archive does not have the catalog folder structure inside.

Fixes # (OCPBUGS-16801)

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

apiVersion: mirror.openshift.io/v1alpha2
kind: ImageSetConfiguration
archiveSize: 1
storageConfig:
  local:
    path: metadata
mirror:
  operators:
  - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.11
    packages:
    - name: netobserv-operator
      channels:
      - name: stable

./bin/oc-mirror -c ./ocpbugs/ocpbugs-16801.yaml file:///home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/ocpbugs-16801

./bin/oc-mirror --from /home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/ocpbugs-16801 docker://localhost:7000 --dest-skip-tls --dest-use-http

Expected Outcome

curl http://localhost:7000/v2/_catalog | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   418  100   418    0     0   137k      0 --:--:-- --:--:-- --:--:--  204k
{
  "repositories": [
    "network-observability/network-observability-console-plugin-rhel9",
    "network-observability/network-observability-ebpf-agent-rhel9",
    "network-observability/network-observability-flowlogs-pipeline-rhel9",
    "network-observability/network-observability-operator-bundle",
    "network-observability/network-observability-rhel9-operator",
    "oc-mirror",
    "openshift4/ose-kube-rbac-proxy",
    "redhat/redhat-operator-index"
  ]
}

[openshift-ci-robot]

@aguidirh: This pull request references Jira Issue OCPBUGS-16801, which is invalid:

• expected the bug to target the "4.16.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

Description

Currently when the field archiveSize on imageSetConfig is set to 1 the mirroring fails for catalogs.

It was happening because the code was not looping in all archives generated to extract all the catalog content. Also, the folder structure was being kept only in the first archive generated.

The fix is to loop in all archives and also to get all files related with the catalogs/ folder, even if the archive does not have the catalog folder structure inside.

Fixes # (OCPBUGS-16801)

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

apiVersion: mirror.openshift.io/v1alpha2
kind: ImageSetConfiguration
archiveSize: 1
storageConfig:
 local:
   path: metadata
mirror:
 operators:
 - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.11
   packages:
   - name: netobserv-operator
     channels:
     - name: stable

./bin/oc-mirror -c ./ocpbugs/ocpbugs-16801.yaml file:///home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/ocpbugs-16801

./bin/oc-mirror --from /home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/ocpbugs-16801 docker://localhost:7000 --dest-skip-tls --dest-use-http

Expected Outcome

curl http://localhost:7000/v2/_catalog | jq
 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
100   418  100   418    0     0   137k      0 --:--:-- --:--:-- --:--:--  204k
{
 "repositories": [
   "network-observability/network-observability-console-plugin-rhel9",
   "network-observability/network-observability-ebpf-agent-rhel9",
   "network-observability/network-observability-flowlogs-pipeline-rhel9",
   "network-observability/network-observability-operator-bundle",
   "network-observability/network-observability-rhel9-operator",
   "oc-mirror",
   "openshift4/ose-kube-rbac-proxy",
   "redhat/redhat-operator-index"
 ]
}

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[aguidirh]

/jira refresh

[openshift-ci-robot]

@aguidirh: This pull request references Jira Issue OCPBUGS-16801, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (4.16.0) matches configured target version for branch (4.16.0)
• bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @zhouying7780

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: aguidirh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [aguidirh]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[lmzuccarelli]

@aguidirh - apart from the unit test - looks good

[openshift-ci]

@aguidirh: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[zhouying7780]

/label qe-approved

[openshift-ci-robot]

@aguidirh: This pull request references Jira Issue OCPBUGS-16801, which is valid.

3 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (4.16.0) matches configured target version for branch (4.16.0)
• bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @zhouying7780

In response to this:

Description

Currently when the field archiveSize on imageSetConfig is set to 1 the mirroring fails for catalogs.

It was happening because the code was not looping in all archives generated to extract all the catalog content. Also, the folder structure was being kept only in the first archive generated.

The fix is to loop in all archives and also to get all files related with the catalogs/ folder, even if the archive does not have the catalog folder structure inside.

Fixes # (OCPBUGS-16801)

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

apiVersion: mirror.openshift.io/v1alpha2
kind: ImageSetConfiguration
archiveSize: 1
storageConfig:
 local:
   path: metadata
mirror:
 operators:
 - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.11
   packages:
   - name: netobserv-operator
     channels:
     - name: stable

./bin/oc-mirror -c ./ocpbugs/ocpbugs-16801.yaml file:///home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/ocpbugs-16801

./bin/oc-mirror --from /home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/ocpbugs-16801 docker://localhost:7000 --dest-skip-tls --dest-use-http

Expected Outcome

curl http://localhost:7000/v2/_catalog | jq
 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
100   418  100   418    0     0   137k      0 --:--:-- --:--:-- --:--:--  204k
{
 "repositories": [
   "network-observability/network-observability-console-plugin-rhel9",
   "network-observability/network-observability-ebpf-agent-rhel9",
   "network-observability/network-observability-flowlogs-pipeline-rhel9",
   "network-observability/network-observability-operator-bundle",
   "network-observability/network-observability-rhel9-operator",
   "oc-mirror",
   "openshift4/ose-kube-rbac-proxy",
   "redhat/redhat-operator-index"
 ]
}

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[lmzuccarelli]

/lgtm

[openshift-ci-robot]

@aguidirh: Jira Issue OCPBUGS-16801: All pull requests linked via external trackers have merged:

• openshift/oc-mirror#773

Jira Issue OCPBUGS-16801 has been moved to the MODIFIED state.

In response to this:

Description

Currently when the field archiveSize on imageSetConfig is set to 1 the mirroring fails for catalogs.

It was happening because the code was not looping in all archives generated to extract all the catalog content. Also, the folder structure was being kept only in the first archive generated.

The fix is to loop in all archives and also to get all files related with the catalogs/ folder, even if the archive does not have the catalog folder structure inside.

Fixes # (OCPBUGS-16801)

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

How Has This Been Tested?

apiVersion: mirror.openshift.io/v1alpha2
kind: ImageSetConfiguration
archiveSize: 1
storageConfig:
 local:
   path: metadata
mirror:
 operators:
 - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.11
   packages:
   - name: netobserv-operator
     channels:
     - name: stable

./bin/oc-mirror -c ./ocpbugs/ocpbugs-16801.yaml file:///home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/ocpbugs-16801

./bin/oc-mirror --from /home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/ocpbugs-16801 docker://localhost:7000 --dest-skip-tls --dest-use-http

Expected Outcome

curl http://localhost:7000/v2/_catalog | jq
 % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                Dload  Upload   Total   Spent    Left  Speed
100   418  100   418    0     0   137k      0 --:--:-- --:--:-- --:--:--  204k
{
 "repositories": [
   "network-observability/network-observability-console-plugin-rhel9",
   "network-observability/network-observability-ebpf-agent-rhel9",
   "network-observability/network-observability-flowlogs-pipeline-rhel9",
   "network-observability/network-observability-operator-bundle",
   "network-observability/network-observability-rhel9-operator",
   "oc-mirror",
   "openshift4/ose-kube-rbac-proxy",
   "redhat/redhat-operator-index"
 ]
}

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[aguidirh]

/cherrypick release-4.15

[openshift-cherrypick-robot]

@aguidirh: new pull request created: #774

In response to this:

/cherrypick release-4.15

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.