Skip to content

Conversation

kuiwang02
Copy link
Contributor

@kuiwang02 kuiwang02 commented Sep 29, 2025

Summary

This PR migrates more OLM v0 test cases from the openshift-tests-private repository to the operator-framework-olm tests-extension. This continues the migration effort to consolidate test cases and improve test organization.

Changes

New Test Spec Files

  • olmv0_allns.go - Tests for OLM handling within all namespaces (155 lines)
  • olmv0_defaultoption.go - Tests for OLM optional and default behaviors (46 lines)
  • olmv0_hypershiftmgmt.go - Tests for OLM on HyperShift management clusters (109 lines)
  • olmv0_microshift.go - Tests for OLM on MicroShift environments (324 lines)
  • olmv0_multins.go - Tests for OLM multi-namespace support (113 lines)
  • olmv0_nonallns.go - Tests for OLM handling within single namespaces (247 lines)

Updated Files

  • olmv0_common.go - Renamed from olmv0.go and updated with common test functions (13 line changes)
  • README.md - Updated with comprehensive test case migration guide and mapping table (21 line changes)
  • Makefile - Enhanced with additional build and test targets (58 line changes)
  • openshift_payload_olmv0.json - Added new test suite configurations (138 lines)
  • filters.go - Added new filter functionality (4 line changes)
  • hypeshift.go - Updated HyperShift utility functions (16 line changes)

Test Case Mapping

The migrated test cases correspond to the following original g.Describe blocks from openshift-tests-private/test/extended/operators/olm.go:

  • [sig-operators] OLM optional → olmv0_defaultoption.go
  • [sig-operators] OLM should → olmv0_defaultoption.go
  • [sig-operators] OLM for an end user handle within a namespace → olmv0_nonallns.go
  • [sig-operators] OLM for an end user handle to support → olmv0_multins.go
  • [sig-operators] OLM for an end user handle within all namespace → olmv0_allns.go
  • [sig-operators] OLM on hypershift → olmv0_hypershiftmgmt.go

Test Result

./openshift-tests run olmv0/extended/candidate/function --monitor watch-namespaces  
  I0929 14:51:49.169265   46301 i18n.go:119] Couldn't find the LC_ALL, LC_MESSAGES or LANG environment variables, defaulting to en_US
  I0929 14:51:49.169422   46301 i18n.go:157] Setting language to default
  I0929 14:51:49.363065   46301 binary.go:86] Found 8057 test specs
  I0929 14:51:49.363776   46301 binary.go:91] 1018 test specs remain, after filtering out k8s
openshift-tests v4.1.0-10044-gf00cea8
...
INFO[0220] Found 0 must-gather tests                    
started: 0/1/11 "[sig-operator][Jira:OLM] OLMvo on hypershift mgmt ROSA-OSD_CCS-HyperShiftMGMT-ConnectedOnly-Author:kuiwang-Medium-45381-Support custom catalogs in hypershift"

started: 0/2/11 "[sig-operator][Jira:OLM] OLMv0 optional should PolarionID:68679-[Skipped:Disconnected]catalogsource with invalid name is created"

started: 0/3/11 "[sig-operator][Jira:OLM] OLMv0 with multi ns PolarionID:22226-[Skipped:Disconnected]the csv without support MultiNamespace fails for og with MultiNamespace"

started: 0/4/11 "[sig-operator][Jira:OLM] OLMv0 on microshift PolarionID:83581-[Skipped:Disconnected]olmv0 networkpolicy on microshift."

started: 0/5/11 "[sig-operator][Jira:OLM] OLMv0 within a namespace PolarionID:24870-[Skipped:Disconnected]can not create csv without operator group"

started: 0/6/11 "[sig-operator][Jira:OLM] OLMv0 should PolarionID:73695-[Skipped:Disconnected]PO is disable"

started: 0/7/11 "[sig-operator][Jira:OLM] OLMv0 on microshift PolarionID:69868-[Skipped:Disconnected]olm microshift install operator with all mode, muilt og error and delete one og to get it installed."

started: 0/8/11 "[sig-operator][Jira:OLM] OLMv0 on microshift PolarionID:69867-[Skipped:Disconnected]deployed in microshift and install one operator with single mode."

started: 0/9/11 "[sig-operator][Jira:OLM] OLMv0 within a namespace PolarionID:37263-[Skipped:Disconnected][Skipped:Proxy]Subscription stays in UpgradePending but InstallPlan not installing [Slow]"

skip [github.com/openshift/operator-framework-olm/tests-extension/test/qe/specs/olmv0_microshift.go:30]: it is not microshift, so skip it.

skipped: (7.3s) 2025-09-29T06:55:46 "[sig-operator][Jira:OLM] OLMv0 on microshift PolarionID:69867-[Skipped:Disconnected]deployed in microshift and install one operator with single mode."

skip [github.com/openshift/operator-framework-olm/tests-extension/test/qe/specs/olmv0_microshift.go:30]: it is not microshift, so skip it.

skipped: (7.2s) 2025-09-29T06:55:46 "[sig-operator][Jira:OLM] OLMv0 on microshift PolarionID:83581-[Skipped:Disconnected]olmv0 networkpolicy on microshift."

skip [github.com/openshift/operator-framework-olm/tests-extension/test/qe/specs/olmv0_microshift.go:30]: it is not microshift, so skip it.

skipped: (7.2s) 2025-09-29T06:55:46 "[sig-operator][Jira:OLM] OLMv0 on microshift PolarionID:69868-[Skipped:Disconnected]olm microshift install operator with all mode, muilt og error and delete one og to get it installed."

skip [github.com/openshift/operator-framework-olm/tests-extension/test/qe/specs/olmv0_common.go:66]: PO is supported in TP only currently, so skip it

skipped: (18.9s) 2025-09-29T06:55:57 "[sig-operator][Jira:OLM] OLMv0 should PolarionID:73695-[Skipped:Disconnected]PO is disable"

skip [github.com/openshift/operator-framework-olm/tests-extension/test/qe/util/hypeshift.go:85]: hypershift command not found in PATH, cannot create kubeconfig for hosted cluster

skipped: (24.6s) 2025-09-29T06:56:03 "[sig-operator][Jira:OLM] OLMvo on hypershift mgmt ROSA-OSD_CCS-HyperShiftMGMT-ConnectedOnly-Author:kuiwang-Medium-45381-Support custom catalogs in hypershift"

passed: (57.3s) 2025-09-29T06:56:36 "[sig-operator][Jira:OLM] OLMv0 optional should PolarionID:68679-[Skipped:Disconnected]catalogsource with invalid name is created"

passed: (1m40s) 2025-09-29T06:57:19 "[sig-operator][Jira:OLM] OLMv0 with multi ns PolarionID:22226-[Skipped:Disconnected]the csv without support MultiNamespace fails for og with MultiNamespace"

passed: (2m31s) 2025-09-29T06:58:09 "[sig-operator][Jira:OLM] OLMv0 within a namespace PolarionID:24870-[Skipped:Disconnected]can not create csv without operator group"

passed: (5m42s) 2025-09-29T07:01:21 "[sig-operator][Jira:OLM] OLMv0 within a namespace PolarionID:37263-[Skipped:Disconnected][Skipped:Proxy]Subscription stays in UpgradePending but InstallPlan not installing [Slow]"

started: 0/10/11 "[sig-operator][Jira:OLM] OLMv0 should PolarionID:22259-[Skipped:Disconnected]marketplace operator CR status on a running cluster[Serial]"

passed: (43.2s) 2025-09-29T07:02:08 "[sig-operator][Jira:OLM] OLMv0 should PolarionID:22259-[Skipped:Disconnected]marketplace operator CR status on a running cluster[Serial]"

started: 0/11/11 "[sig-operator][Jira:OLM] OLMv0 within all namespace PolarionID:21418-PolarionID:25679-[Skipped:Disconnected]Cluster resource created and deleted correctly [Serial]"

passed: (2m9s) 2025-09-29T07:04:21 "[sig-operator][Jira:OLM] OLMv0 within all namespace PolarionID:21418-PolarionID:25679-[Skipped:Disconnected]Cluster resource created and deleted correctly [Serial]"

Shutting down the monitor
Collecting data.
INFO[0752] Starting CollectData for all monitor tests   
INFO[0752]   Starting CollectData for [Monitor:watch-namespaces][Jira:"Test Framework"] monitor test watch-namespaces collection 
INFO[0752]   Finished CollectData for [Monitor:watch-namespaces][Jira:"Test Framework"] monitor test watch-namespaces collection 
INFO[0752] Finished CollectData for all monitor tests   
Computing intervals.
Evaluating tests.
Cleaning up.
INFO[0752] beginning cleanup                             monitorTest=watch-namespaces
Serializing results.
Writing to storage.
  m.startTime = 2025-09-29 14:55:29.936796 +0800 CST m=+220.841553001
  m.stopTime  = 2025-09-29 15:04:21.87746 +0800 CST m=+752.804876709
Processing monitorTest: watch-namespaces
  finalIntervals size = 22
  first interval time: From = 2025-09-29 14:55:29.945182 +0800 CST m=+220.849938209; To = 2025-09-29 14:55:29.945182 +0800 CST m=+220.849938209
  last interval time: From = 2025-09-29 15:04:21.876823 +0800 CST m=+752.804239626; To = 2025-09-29 15:04:21.876823 +0800 CST m=+752.804239626
Writing junits.
Writing JUnit report to e2e-monitor-tests__20250929-065228.xml
6 pass, 0 flaky, 5 skip (11m54s)
INFO[0756] Successfully removed tmpDir /var/folders/wl/t8n9cyz511q96jx5xvjfwkn40000gn/T/openshift-tests1051731603 

Assisted-by: Claude Code

@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 29, 2025
@kuiwang02
Copy link
Contributor Author

kuiwang02 commented Sep 29, 2025

/retitle OCPQE-30679:migrate more cases from tests-private to make layout

@openshift-ci openshift-ci bot changed the title [WIP]migrate more cases from tests-private OCPQE-30679:migrate more cases from tests-private to make layout Sep 29, 2025
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 29, 2025
@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Sep 29, 2025
@openshift-ci-robot
Copy link

openshift-ci-robot commented Sep 29, 2025

@kuiwang02: This pull request references OCPQE-30679 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the sub-task to target the "4.21.0" version, but no target version was set.

In response to this:

Summary

This PR migrates more OLM v0 test cases from the openshift-tests-private repository to the operator-framework-olm tests-extension. This continues the migration effort to consolidate test cases and improve test organization.

Changes

New Test Spec Files

  • olmv0_allns.go - Tests for OLM handling within all namespaces (155 lines)
  • olmv0_defaultoption.go - Tests for OLM optional and default behaviors (46 lines)
  • olmv0_hypershiftmgmt.go - Tests for OLM on HyperShift management clusters (109 lines)
  • olmv0_microshift.go - Tests for OLM on MicroShift environments (324 lines)
  • olmv0_multins.go - Tests for OLM multi-namespace support (113 lines)
  • olmv0_nonallns.go - Tests for OLM handling within single namespaces (247 lines)

Updated Files

  • olmv0_common.go - Renamed from olmv0.go and updated with common test functions (13 line changes)
  • README.md - Updated with comprehensive test case migration guide and mapping table (21 line changes)
  • Makefile - Enhanced with additional build and test targets (58 line changes)
  • openshift_payload_olmv0.json - Added new test suite configurations (138 lines)
  • filters.go - Added new filter functionality (4 line changes)
  • hypeshift.go - Updated HyperShift utility functions (16 line changes)

Test Case Mapping

The migrated test cases correspond to the following original g.Describe blocks from openshift-tests-private/test/extended/operators/olm.go:

  • [sig-operators] OLM optional → olmv0_defaultoption.go
  • [sig-operators] OLM should → olmv0_defaultoption.go
  • [sig-operators] OLM for an end user handle within a namespace → olmv0_nonallns.go
  • [sig-operators] OLM for an end user handle to support → olmv0_multins.go
  • [sig-operators] OLM for an end user handle within all namespace → olmv0_allns.go
  • [sig-operators] OLM on hypershift → olmv0_hypershiftmgmt.go

Test Result

./openshift-tests run olmv0/extended/candidate/function --monitor watch-namespaces  
 I0929 14:51:49.169265   46301 i18n.go:119] Couldn't find the LC_ALL, LC_MESSAGES or LANG environment variables, defaulting to en_US
 I0929 14:51:49.169422   46301 i18n.go:157] Setting language to default
 I0929 14:51:49.363065   46301 binary.go:86] Found 8057 test specs
 I0929 14:51:49.363776   46301 binary.go:91] 1018 test specs remain, after filtering out k8s
openshift-tests v4.1.0-10044-gf00cea8
...
INFO[0220] Found 0 must-gather tests                    
started: 0/1/11 "[sig-operator][Jira:OLM] OLMvo on hypershift mgmt ROSA-OSD_CCS-HyperShiftMGMT-ConnectedOnly-Author:kuiwang-Medium-45381-Support custom catalogs in hypershift"

started: 0/2/11 "[sig-operator][Jira:OLM] OLMv0 optional should PolarionID:68679-[Skipped:Disconnected]catalogsource with invalid name is created"

started: 0/3/11 "[sig-operator][Jira:OLM] OLMv0 with multi ns PolarionID:22226-[Skipped:Disconnected]the csv without support MultiNamespace fails for og with MultiNamespace"

started: 0/4/11 "[sig-operator][Jira:OLM] OLMv0 on microshift PolarionID:83581-[Skipped:Disconnected]olmv0 networkpolicy on microshift."

started: 0/5/11 "[sig-operator][Jira:OLM] OLMv0 within a namespace PolarionID:24870-[Skipped:Disconnected]can not create csv without operator group"

started: 0/6/11 "[sig-operator][Jira:OLM] OLMv0 should PolarionID:73695-[Skipped:Disconnected]PO is disable"

started: 0/7/11 "[sig-operator][Jira:OLM] OLMv0 on microshift PolarionID:69868-[Skipped:Disconnected]olm microshift install operator with all mode, muilt og error and delete one og to get it installed."

started: 0/8/11 "[sig-operator][Jira:OLM] OLMv0 on microshift PolarionID:69867-[Skipped:Disconnected]deployed in microshift and install one operator with single mode."

started: 0/9/11 "[sig-operator][Jira:OLM] OLMv0 within a namespace PolarionID:37263-[Skipped:Disconnected][Skipped:Proxy]Subscription stays in UpgradePending but InstallPlan not installing [Slow]"

skip [github.com/openshift/operator-framework-olm/tests-extension/test/qe/specs/olmv0_microshift.go:30]: it is not microshift, so skip it.

skipped: (7.3s) 2025-09-29T06:55:46 "[sig-operator][Jira:OLM] OLMv0 on microshift PolarionID:69867-[Skipped:Disconnected]deployed in microshift and install one operator with single mode."

skip [github.com/openshift/operator-framework-olm/tests-extension/test/qe/specs/olmv0_microshift.go:30]: it is not microshift, so skip it.

skipped: (7.2s) 2025-09-29T06:55:46 "[sig-operator][Jira:OLM] OLMv0 on microshift PolarionID:83581-[Skipped:Disconnected]olmv0 networkpolicy on microshift."

skip [github.com/openshift/operator-framework-olm/tests-extension/test/qe/specs/olmv0_microshift.go:30]: it is not microshift, so skip it.

skipped: (7.2s) 2025-09-29T06:55:46 "[sig-operator][Jira:OLM] OLMv0 on microshift PolarionID:69868-[Skipped:Disconnected]olm microshift install operator with all mode, muilt og error and delete one og to get it installed."

skip [github.com/openshift/operator-framework-olm/tests-extension/test/qe/specs/olmv0_common.go:66]: PO is supported in TP only currently, so skip it

skipped: (18.9s) 2025-09-29T06:55:57 "[sig-operator][Jira:OLM] OLMv0 should PolarionID:73695-[Skipped:Disconnected]PO is disable"

skip [github.com/openshift/operator-framework-olm/tests-extension/test/qe/util/hypeshift.go:85]: hypershift command not found in PATH, cannot create kubeconfig for hosted cluster

skipped: (24.6s) 2025-09-29T06:56:03 "[sig-operator][Jira:OLM] OLMvo on hypershift mgmt ROSA-OSD_CCS-HyperShiftMGMT-ConnectedOnly-Author:kuiwang-Medium-45381-Support custom catalogs in hypershift"

passed: (57.3s) 2025-09-29T06:56:36 "[sig-operator][Jira:OLM] OLMv0 optional should PolarionID:68679-[Skipped:Disconnected]catalogsource with invalid name is created"

passed: (1m40s) 2025-09-29T06:57:19 "[sig-operator][Jira:OLM] OLMv0 with multi ns PolarionID:22226-[Skipped:Disconnected]the csv without support MultiNamespace fails for og with MultiNamespace"

passed: (2m31s) 2025-09-29T06:58:09 "[sig-operator][Jira:OLM] OLMv0 within a namespace PolarionID:24870-[Skipped:Disconnected]can not create csv without operator group"

passed: (5m42s) 2025-09-29T07:01:21 "[sig-operator][Jira:OLM] OLMv0 within a namespace PolarionID:37263-[Skipped:Disconnected][Skipped:Proxy]Subscription stays in UpgradePending but InstallPlan not installing [Slow]"

started: 0/10/11 "[sig-operator][Jira:OLM] OLMv0 should PolarionID:22259-[Skipped:Disconnected]marketplace operator CR status on a running cluster[Serial]"

passed: (43.2s) 2025-09-29T07:02:08 "[sig-operator][Jira:OLM] OLMv0 should PolarionID:22259-[Skipped:Disconnected]marketplace operator CR status on a running cluster[Serial]"

started: 0/11/11 "[sig-operator][Jira:OLM] OLMv0 within all namespace PolarionID:21418-PolarionID:25679-[Skipped:Disconnected]Cluster resource created and deleted correctly [Serial]"

passed: (2m9s) 2025-09-29T07:04:21 "[sig-operator][Jira:OLM] OLMv0 within all namespace PolarionID:21418-PolarionID:25679-[Skipped:Disconnected]Cluster resource created and deleted correctly [Serial]"

Shutting down the monitor
Collecting data.
INFO[0752] Starting CollectData for all monitor tests   
INFO[0752]   Starting CollectData for [Monitor:watch-namespaces][Jira:"Test Framework"] monitor test watch-namespaces collection 
INFO[0752]   Finished CollectData for [Monitor:watch-namespaces][Jira:"Test Framework"] monitor test watch-namespaces collection 
INFO[0752] Finished CollectData for all monitor tests   
Computing intervals.
Evaluating tests.
Cleaning up.
INFO[0752] beginning cleanup                             monitorTest=watch-namespaces
Serializing results.
Writing to storage.
 m.startTime = 2025-09-29 14:55:29.936796 +0800 CST m=+220.841553001
 m.stopTime  = 2025-09-29 15:04:21.87746 +0800 CST m=+752.804876709
Processing monitorTest: watch-namespaces
 finalIntervals size = 22
 first interval time: From = 2025-09-29 14:55:29.945182 +0800 CST m=+220.849938209; To = 2025-09-29 14:55:29.945182 +0800 CST m=+220.849938209
 last interval time: From = 2025-09-29 15:04:21.876823 +0800 CST m=+752.804239626; To = 2025-09-29 15:04:21.876823 +0800 CST m=+752.804239626
Writing junits.
Writing JUnit report to e2e-monitor-tests__20250929-065228.xml
6 pass, 0 flaky, 5 skip (11m54s)
INFO[0756] Successfully removed tmpDir /var/folders/wl/t8n9cyz511q96jx5xvjfwkn40000gn/T/openshift-tests1051731603 

Assisted-by: Claude Code

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Copy link
Contributor

openshift-ci bot commented Sep 29, 2025

@kuiwang02: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/okd-scos-e2e-aws-ovn f54d2a8 link false /test okd-scos-e2e-aws-ovn

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@kuiwang02
Copy link
Contributor Author

/verified by @kuiwang02

@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Sep 30, 2025
@openshift-ci-robot
Copy link

@kuiwang02: This PR has been marked as verified by @kuiwang02.

In response to this:

/verified by @kuiwang02

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@jianzhangbjz
Copy link
Contributor

/approve
/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Sep 30, 2025
Copy link
Contributor

openshift-ci bot commented Sep 30, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jianzhangbjz, kuiwang02

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 30, 2025
@openshift-merge-bot openshift-merge-bot bot merged commit 003fcb2 into openshift:main Sep 30, 2025
14 of 15 checks passed
@kuiwang02 kuiwang02 deleted the migratecases branch September 30, 2025 02:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. verified Signifies that the PR passed pre-merge verification criteria
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants