Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CLID-7: feat: load operator catalog using LoadFS #799

Merged
merged 1 commit into from Feb 19, 2024

Conversation

aguidirh
Copy link
Contributor

Description

As part of the mirroring process it is necessary to be able to filter the content to be mirrored by using the ImageSetConfig. This implementation cover all scenarios specified in the operator-filtering-study.md.

This implementation load the catalog using operator framework LoadFS mechanism and remove types duplication between oc-mirror and operator framework.

Fixes # (CLID-7)

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

First step: Choose one of the image set config provided below and add this content to your .yaml file

Second step: run the mirrorToDisk command below:

./bin/oc-mirror -c /home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/alex-isc/operator-filtering-CLID-7.yaml file:///home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/clid-7 --v2

Third step: run the diskToMirror command below:

./bin/oc-mirror -c /home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/alex-isc/operator-filtering-CLID-14.yaml --from file:///home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/clid-14 docker://localhost:7000 --v2

Last step: check if the images are in your target registry

All catalog filtering scenarios are covered by the following ImageSetConfigs:

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v1alpha2
mirror:
  operators:
    - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.14

Expected Outcome: For each package in the catalog, 1 bundle, corresponding to the head version of the default channel for that package

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v1alpha2
mirror:
  operators:
    - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.14
      packages: 
        - name: devworkspace-operator

Expected Outcome: 1 bundle, corresponding to the head version of the default channel for that package

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v1alpha2
mirror:
  operators:
    - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.14
      packages: 
        - name: devworkspace-operator
          minVersion: "0.18.1"

Expected Outcome: all bundles in the default channel, from minVersion, up to channel head for that package (not relying of shortest path from upgrade graph)

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v1alpha2
mirror:
  operators:
    - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.14
      packages: 
        - name: devworkspace-operator
          maxVersion: "0.18.1"

Expected Outcome: all bundles in the default channel, that are lower than maxVersion for that package

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v1alpha2
mirror:
  operators:
    - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.14
      packages:
        - name: devworkspace-operator
          minVersion: 0.16.0
          maxVersion: 0.17.0

Expected Outcome: all bundles in the default channel, between minVersion and maxVersion for that package. Head of channel is not included, even if multiple channels are included in the filtering

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v1alpha2
mirror:
  operators:
    - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.14
      packages:
        - name: 3scale-operator
          channels:
            - name: threescale-mas
              minVersion: 0.11.7

Expected Outcome: within the selected channel of that package, all version starting minVersion up to channel head (not relying of shortest path from upgrade graph)

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v1alpha2
mirror:
  operators:
    - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.14
      packages:
        - name: 3scale-operator
          channels:
            - name: threescale-mas

Expected Outcome: head bundle for the selected channel of that package

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v1alpha2
mirror:
  operators:
    - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.14
      packages:
        - name: 3scale-operator
          channels:
            - name: threescale-2.14
            - name: threescale-mas

Expected Outcome: head bundle for the each selected channel of that package

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v1alpha2
mirror:
  operators:
    - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.14
      packages:
        - name: 3scale-operator
          channels:
            - name: threescale-mas
              maxVersion: 0.11.1

Expected Outcome: within the selected channel of that package, all versions up to maxVersion (not relying of shortest path from upgrade graph): Head of channel is not included, even if multiple channels are included in the filtering

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v1alpha2
mirror:
  operators:
    - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.14
      packages:
        - name: 3scale-operator
          channels:
            - name: threescale-mas
              minVersion: 0.11.1
              maxVersion: 0.11.3

Expected Outcome: within the selected channel of that package, all versions between minVersion and maxVersion (not relying of shortest path from upgrade graph): Head of channel is not included, even if multiple channels are included in the filtering

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v1alpha2
mirror:
  operators:
    - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.14
      packages:
        - name: 3scale-operator
          channels:
            - name: threescale-2.11
              minVersion: 0.8.1
              maxVersion: 0.8.3
          minVersion: 0.8.1

Expected Outcome: Error: filtering by channel and by package min/max should not be allowed

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v1alpha2
mirror:
  operators:
    - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.14
      full: true

Expected Outcome: all bundles of all channels of the specified catalog

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v1alpha2
mirror:
  operators:
    - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.14
      full: true
      packages: 
        - name: devworkspace-operator

Expected Outcome: all bundles of all channels for the packages specified

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v1alpha2
mirror:
  operators:
    - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.14
      full: true
      packages: 
        - name: 3scale-operator
          channels:
            - name: threescale-mas

Expected Outcome: # all bundles for the packages and channels specified

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Feb 15, 2024
@openshift-ci-robot
Copy link

openshift-ci-robot commented Feb 15, 2024

@aguidirh: This pull request references CLID-7 which is a valid jira issue.

In response to this:

Description

As part of the mirroring process it is necessary to be able to filter the content to be mirrored by using the ImageSetConfig. This implementation cover all scenarios specified in the operator-filtering-study.md.

This implementation load the catalog using operator framework LoadFS mechanism and remove types duplication between oc-mirror and operator framework.

Fixes # (CLID-7)

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

First step: Choose one of the image set config provided below and add this content to your .yaml file

Second step: run the mirrorToDisk command below:

./bin/oc-mirror -c /home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/alex-isc/operator-filtering-CLID-7.yaml file:///home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/clid-7 --v2

Third step: run the diskToMirror command below:

./bin/oc-mirror -c /home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/alex-isc/operator-filtering-CLID-14.yaml --from file:///home/aguidi/go/src/github.com/aguidirh/oc-mirror/alex-tests/clid-14 docker://localhost:7000 --v2

Last step: check if the images are in your target registry

All catalog filtering scenarios are covered by the following ImageSetConfigs:

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v1alpha2
mirror:
 operators:
   - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.14

Expected Outcome: For each package in the catalog, 1 bundle, corresponding to the head version of the default channel for that package

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v1alpha2
mirror:
 operators:
   - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.14
     packages: 
       - name: devworkspace-operator

Expected Outcome: 1 bundle, corresponding to the head version of the default channel for that package

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v1alpha2
mirror:
 operators:
   - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.14
     packages: 
       - name: devworkspace-operator
         minVersion: "0.18.1"

Expected Outcome: all bundles in the default channel, from minVersion, up to channel head for that package (not relying of shortest path from upgrade graph)

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v1alpha2
mirror:
 operators:
   - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.14
     packages: 
       - name: devworkspace-operator
         maxVersion: "0.18.1"

Expected Outcome: all bundles in the default channel, that are lower than maxVersion for that package

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v1alpha2
mirror:
 operators:
   - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.14
     packages:
       - name: devworkspace-operator
         minVersion: 0.16.0
         maxVersion: 0.17.0

Expected Outcome: all bundles in the default channel, between minVersion and maxVersion for that package. Head of channel is not included, even if multiple channels are included in the filtering

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v1alpha2
mirror:
 operators:
   - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.14
     packages:
       - name: 3scale-operator
         channels:
           - name: threescale-mas
             minVersion: 0.11.7

Expected Outcome: within the selected channel of that package, all version starting minVersion up to channel head (not relying of shortest path from upgrade graph)

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v1alpha2
mirror:
 operators:
   - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.14
     packages:
       - name: 3scale-operator
         channels:
           - name: threescale-mas

Expected Outcome: head bundle for the selected channel of that package

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v1alpha2
mirror:
 operators:
   - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.14
     packages:
       - name: 3scale-operator
         channels:
           - name: threescale-2.14
           - name: threescale-mas

Expected Outcome: head bundle for the each selected channel of that package

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v1alpha2
mirror:
 operators:
   - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.14
     packages:
       - name: 3scale-operator
         channels:
           - name: threescale-mas
             maxVersion: 0.11.1

Expected Outcome: within the selected channel of that package, all versions up to maxVersion (not relying of shortest path from upgrade graph): Head of channel is not included, even if multiple channels are included in the filtering

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v1alpha2
mirror:
 operators:
   - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.14
     packages:
       - name: 3scale-operator
         channels:
           - name: threescale-mas
             minVersion: 0.11.1
             maxVersion: 0.11.3

Expected Outcome: within the selected channel of that package, all versions between minVersion and maxVersion (not relying of shortest path from upgrade graph): Head of channel is not included, even if multiple channels are included in the filtering

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v1alpha2
mirror:
 operators:
   - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.14
     packages:
       - name: 3scale-operator
         channels:
           - name: threescale-2.11
             minVersion: 0.8.1
             maxVersion: 0.8.3
         minVersion: 0.8.1

Expected Outcome: Error: filtering by channel and by package min/max should not be allowed

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v1alpha2
mirror:
 operators:
   - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.14
     full: true

Expected Outcome: all bundles of all channels of the specified catalog

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v1alpha2
mirror:
 operators:
   - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.14
     full: true
     packages: 
       - name: devworkspace-operator

Expected Outcome: all bundles of all channels for the packages specified

kind: ImageSetConfiguration
apiVersion: mirror.openshift.io/v1alpha2
mirror:
 operators:
   - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.14
     full: true
     packages: 
       - name: 3scale-operator
         channels:
           - name: threescale-mas

Expected Outcome: # all bundles for the packages and channels specified

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 15, 2024
Copy link
Contributor

@lmzuccarelli lmzuccarelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aguidirh - I'm slowly beginning to hate go.mod changes - your changes -> 10 files , the rest -> 278 with version bump

v2/pkg/api/v1alpha3/types.go Show resolved Hide resolved
v2/pkg/manifest/const.go Show resolved Hide resolved
v2/pkg/manifest/oci-manifest_test.go Show resolved Hide resolved
@lmzuccarelli
Copy link
Contributor

lmzuccarelli commented Feb 19, 2024

@aguidirh - I haven't tested all the scenarios, I'll do the easiest one (or 2) later today (bandwidth and internet connection permitting) :) , leave the rest for QE

Copy link
Contributor

@sherine-k sherine-k left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work @aguidirh
I noted just a few nits here and there..
thank you so much

v2/pkg/api/v1alpha3/types.go Show resolved Hide resolved
v2/pkg/manifest/oci-manifest.go Outdated Show resolved Hide resolved
v2/pkg/manifest/oci-manifest.go Outdated Show resolved Hide resolved
v2/pkg/manifest/oci-manifest.go Outdated Show resolved Hide resolved
v2/pkg/manifest/oci-manifest.go Outdated Show resolved Hide resolved
Copy link

openshift-ci bot commented Feb 19, 2024

@aguidirh: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

Copy link
Contributor

@sherine-k sherine-k left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@sherine-k
Copy link
Contributor

/label acknowledge-critical-fixes-only

@openshift-ci openshift-ci bot added the acknowledge-critical-fixes-only Indicates if the issuer of the label is OK with the policy. label Feb 19, 2024
@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Feb 19, 2024
Copy link

openshift-ci bot commented Feb 19, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: aguidirh, sherine-k

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-bot openshift-merge-bot bot merged commit 6320ebd into openshift:main Feb 19, 2024
5 checks passed
@openshift-bot
Copy link
Contributor

[ART PR BUILD NOTIFIER]

This PR has been included in build oc-mirror-plugin-container-v4.16.0-202402191709.p0.g6320ebd.assembly.stream.el9 for distgit oc-mirror-plugin.
All builds following this will include this PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
acknowledge-critical-fixes-only Indicates if the issuer of the label is OK with the policy. approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants