Skip to content

Conversation

kuiwang02
Copy link
Contributor

Summary

Migrate additional test cases from tests-private repository and enhance test suites with improved filtering capabilities for OLM v1 testing.

Changes

  • Test Migration: Added 1200+ lines of new test cases migrated from tests-private to olmv1_ce.go
  • Filter Enhancement: Implemented new filtering utilities in test/qe/util/filters/filters.go
  • Main Command Updates: Enhanced the main test command with additional functionality
  • Test Data Updates: Updated OLM test data configuration files
  • Payload Configuration: Added comprehensive OpenShift payload configuration for OLM v1 testing

Result

./openshift-tests run olmv1/extended/candidate/function --monitor watch-namespaces  
  I0926 11:27:20.972742    9196 i18n.go:119] Couldn't find the LC_ALL, LC_MESSAGES or LANG environment variables, defaulting to en_US
  I0926 11:27:20.972849    9196 i18n.go:157] Setting language to default
openshift-tests v4.1.0-10015-g8157b72
...
INFO[0226] Found 0 must-gather tests                    
started: 0/1/15 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:74618-[Skipped:Disconnected]ClusterExtension supports simple registry vzero bundles only"

started: 0/2/15 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:81538-[Skipped:Disconnected]preflight check on permission on allns mode"

started: 0/3/15 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:75492-[Skipped:Disconnected]cluster extension can not be installed with wrong sa or insufficient permission sa"

started: 0/4/15 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:81664-[Skipped:Disconnected]preflight check on permission on own ns mode"

started: 0/5/15 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:76983-[Skipped:Disconnected]install index and bundle from private image[Slow]"

started: 0/6/15 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:68936-[Skipped:Disconnected]cluster extension can not be installed with insufficient permission sa for operand"

started: 0/7/15 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:68937-[Skipped:Disconnected]cluster extension can not be installed with insufficient permission sa for operand rbac object"

started: 0/8/15 "[sig-olmv1][Jira:OLM] clustercatalog PolarionID:69242-[Skipped:Disconnected]Catalogd deprecated package/bundlemetadata/catalogmetadata from clustercatalog CR"

started: 0/9/15 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:75493-[Skipped:Disconnected]cluster extension can be installed with enough permission sa"

started: 0/10/15 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:81696-[Skipped:Disconnected]preflight check on permission on single ns mode"

passed: (20.4s) 2025-09-26T03:31:39 "[sig-olmv1][Jira:OLM] clustercatalog PolarionID:69242-[Skipped:Disconnected]Catalogd deprecated package/bundlemetadata/catalogmetadata from clustercatalog CR"

passed: (2m7s) 2025-09-26T03:33:27 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:76983-[Skipped:Disconnected]install index and bundle from private image[Slow]"

passed: (2m20s) 2025-09-26T03:33:38 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:81538-[Skipped:Disconnected]preflight check on permission on allns mode"

passed: (2m39s) 2025-09-26T03:33:58 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:81664-[Skipped:Disconnected]preflight check on permission on own ns mode"

passed: (3m13s) 2025-09-26T03:34:31 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:68936-[Skipped:Disconnected]cluster extension can not be installed with insufficient permission sa for operand"

passed: (3m16s) 2025-09-26T03:34:35 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:81696-[Skipped:Disconnected]preflight check on permission on single ns mode"

passed: (3m26s) 2025-09-26T03:34:46 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:68937-[Skipped:Disconnected]cluster extension can not be installed with insufficient permission sa for operand rbac object"

passed: (3m37s) 2025-09-26T03:34:56 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:75493-[Skipped:Disconnected]cluster extension can be installed with enough permission sa"

passed: (3m46s) 2025-09-26T03:35:06 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:74618-[Skipped:Disconnected]ClusterExtension supports simple registry vzero bundles only"

passed: (3m51s) 2025-09-26T03:35:10 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:75492-[Skipped:Disconnected]cluster extension can not be installed with wrong sa or insufficient permission sa"

started: 0/11/15 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:78300-[Skipped:Disconnected]validation of container images using sigstore signatures with different policy [Serial][Disruptive][Slow]"

passed: (11m27s) 2025-09-26T03:46:47 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:78300-[Skipped:Disconnected]validation of container images using sigstore signatures with different policy [Serial][Disruptive][Slow]"

started: 0/12/15 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:76843-[Skipped:Disconnected]support disc with icsp[Timeout:30m] [Serial][Disruptive][Slow]"

passed: (20m17s) 2025-09-26T04:07:12 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:76843-[Skipped:Disconnected]support disc with icsp[Timeout:30m] [Serial][Disruptive][Slow]"

started: 0/13/15 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:76844-[Skipped:Disconnected]support disc with itms and idms[Timeout:30m] [Serial][Disruptive][Slow]"

skip [github.com/openshift/operator-framework-operator-controller/openshift/tests-extension/test/qe/specs/olmv1_ce.go:1037]: ImageContentSourcePolicy already exists in cluster. This test requires ITMS/IDMS-only environment.

skipped: (16.5s) 2025-09-26T04:07:36 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:76844-[Skipped:Disconnected]support disc with itms and idms[Timeout:30m] [Serial][Disruptive][Slow]"

started: 0/14/15 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:76985-[Skipped:Disconnected]authfile is updated automatically[Timeout:30m] [Serial][Disruptive][Slow]"

passed: (9m38s) 2025-09-26T04:17:23 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:76985-[Skipped:Disconnected]authfile is updated automatically[Timeout:30m] [Serial][Disruptive][Slow]"

started: 0/15/15 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:78193-[Skipped:Disconnected]Runtime validation of container images using sigstore signatures [Serial][Disruptive][Slow]"

passed: (11m57s) 2025-09-26T04:29:27 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:78193-[Skipped:Disconnected]Runtime validation of container images using sigstore signatures [Serial][Disruptive][Slow]"

Shutting down the monitor
Collecting data.
INFO[3726] Starting CollectData for all monitor tests   
INFO[3726]   Starting CollectData for [Monitor:watch-namespaces][Jira:"Test Framework"] monitor test watch-namespaces collection 
INFO[3726]   Finished CollectData for [Monitor:watch-namespaces][Jira:"Test Framework"] monitor test watch-namespaces collection 
INFO[3726] Finished CollectData for all monitor tests   
Computing intervals.
Evaluating tests.
Cleaning up.
INFO[3726] beginning cleanup                             monitorTest=watch-namespaces
Serializing results.
Writing to storage.
  m.startTime = 2025-09-26 11:31:07.177184 +0800 CST m=+226.458576459
  m.stopTime  = 2025-09-26 12:29:27.476377 +0800 CST m=+3726.772710917
Processing monitorTest: watch-namespaces
  finalIntervals size = 30
  first interval time: From = 2025-09-26 11:31:07.186684 +0800 CST m=+226.468076251; To = 2025-09-26 11:31:07.186684 +0800 CST m=+226.468076251
  last interval time: From = 2025-09-26 12:29:27.475089 +0800 CST m=+3726.771423001; To = 2025-09-26 12:29:27.475089 +0800 CST m=+3726.771423001
Writing junits.
Writing JUnit report to e2e-monitor-tests__20250926-032805.xml
14 pass, 0 flaky, 1 skip (1h1m22s)
INFO[3729] Successfully removed tmpDir /var/folders/wl/t8n9cyz511q96jx5xvjfwkn40000gn/T/openshift-tests1409893501 

Assisted-by: Claude Code

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Sep 26, 2025
@openshift-ci-robot
Copy link

openshift-ci-robot commented Sep 26, 2025

@kuiwang02: This pull request references OCPQE-30671 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the sub-task to target the "4.21.0" version, but no target version was set.

In response to this:

Summary

Migrate additional test cases from tests-private repository and enhance test suites with improved filtering capabilities for OLM v1 testing.

Changes

  • Test Migration: Added 1200+ lines of new test cases migrated from tests-private to olmv1_ce.go
  • Filter Enhancement: Implemented new filtering utilities in test/qe/util/filters/filters.go
  • Main Command Updates: Enhanced the main test command with additional functionality
  • Test Data Updates: Updated OLM test data configuration files
  • Payload Configuration: Added comprehensive OpenShift payload configuration for OLM v1 testing

Result

./openshift-tests run olmv1/extended/candidate/function --monitor watch-namespaces  
 I0926 11:27:20.972742    9196 i18n.go:119] Couldn't find the LC_ALL, LC_MESSAGES or LANG environment variables, defaulting to en_US
 I0926 11:27:20.972849    9196 i18n.go:157] Setting language to default
openshift-tests v4.1.0-10015-g8157b72
...
INFO[0226] Found 0 must-gather tests                    
started: 0/1/15 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:74618-[Skipped:Disconnected]ClusterExtension supports simple registry vzero bundles only"

started: 0/2/15 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:81538-[Skipped:Disconnected]preflight check on permission on allns mode"

started: 0/3/15 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:75492-[Skipped:Disconnected]cluster extension can not be installed with wrong sa or insufficient permission sa"

started: 0/4/15 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:81664-[Skipped:Disconnected]preflight check on permission on own ns mode"

started: 0/5/15 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:76983-[Skipped:Disconnected]install index and bundle from private image[Slow]"

started: 0/6/15 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:68936-[Skipped:Disconnected]cluster extension can not be installed with insufficient permission sa for operand"

started: 0/7/15 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:68937-[Skipped:Disconnected]cluster extension can not be installed with insufficient permission sa for operand rbac object"

started: 0/8/15 "[sig-olmv1][Jira:OLM] clustercatalog PolarionID:69242-[Skipped:Disconnected]Catalogd deprecated package/bundlemetadata/catalogmetadata from clustercatalog CR"

started: 0/9/15 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:75493-[Skipped:Disconnected]cluster extension can be installed with enough permission sa"

started: 0/10/15 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:81696-[Skipped:Disconnected]preflight check on permission on single ns mode"

passed: (20.4s) 2025-09-26T03:31:39 "[sig-olmv1][Jira:OLM] clustercatalog PolarionID:69242-[Skipped:Disconnected]Catalogd deprecated package/bundlemetadata/catalogmetadata from clustercatalog CR"

passed: (2m7s) 2025-09-26T03:33:27 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:76983-[Skipped:Disconnected]install index and bundle from private image[Slow]"

passed: (2m20s) 2025-09-26T03:33:38 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:81538-[Skipped:Disconnected]preflight check on permission on allns mode"

passed: (2m39s) 2025-09-26T03:33:58 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:81664-[Skipped:Disconnected]preflight check on permission on own ns mode"

passed: (3m13s) 2025-09-26T03:34:31 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:68936-[Skipped:Disconnected]cluster extension can not be installed with insufficient permission sa for operand"

passed: (3m16s) 2025-09-26T03:34:35 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:81696-[Skipped:Disconnected]preflight check on permission on single ns mode"

passed: (3m26s) 2025-09-26T03:34:46 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:68937-[Skipped:Disconnected]cluster extension can not be installed with insufficient permission sa for operand rbac object"

passed: (3m37s) 2025-09-26T03:34:56 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:75493-[Skipped:Disconnected]cluster extension can be installed with enough permission sa"

passed: (3m46s) 2025-09-26T03:35:06 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:74618-[Skipped:Disconnected]ClusterExtension supports simple registry vzero bundles only"

passed: (3m51s) 2025-09-26T03:35:10 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:75492-[Skipped:Disconnected]cluster extension can not be installed with wrong sa or insufficient permission sa"

started: 0/11/15 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:78300-[Skipped:Disconnected]validation of container images using sigstore signatures with different policy [Serial][Disruptive][Slow]"

passed: (11m27s) 2025-09-26T03:46:47 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:78300-[Skipped:Disconnected]validation of container images using sigstore signatures with different policy [Serial][Disruptive][Slow]"

started: 0/12/15 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:76843-[Skipped:Disconnected]support disc with icsp[Timeout:30m] [Serial][Disruptive][Slow]"

passed: (20m17s) 2025-09-26T04:07:12 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:76843-[Skipped:Disconnected]support disc with icsp[Timeout:30m] [Serial][Disruptive][Slow]"

started: 0/13/15 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:76844-[Skipped:Disconnected]support disc with itms and idms[Timeout:30m] [Serial][Disruptive][Slow]"

skip [github.com/openshift/operator-framework-operator-controller/openshift/tests-extension/test/qe/specs/olmv1_ce.go:1037]: ImageContentSourcePolicy already exists in cluster. This test requires ITMS/IDMS-only environment.

skipped: (16.5s) 2025-09-26T04:07:36 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:76844-[Skipped:Disconnected]support disc with itms and idms[Timeout:30m] [Serial][Disruptive][Slow]"

started: 0/14/15 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:76985-[Skipped:Disconnected]authfile is updated automatically[Timeout:30m] [Serial][Disruptive][Slow]"

passed: (9m38s) 2025-09-26T04:17:23 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:76985-[Skipped:Disconnected]authfile is updated automatically[Timeout:30m] [Serial][Disruptive][Slow]"

started: 0/15/15 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:78193-[Skipped:Disconnected]Runtime validation of container images using sigstore signatures [Serial][Disruptive][Slow]"

passed: (11m57s) 2025-09-26T04:29:27 "[sig-olmv1][Jira:OLM] clusterextension PolarionID:78193-[Skipped:Disconnected]Runtime validation of container images using sigstore signatures [Serial][Disruptive][Slow]"

Shutting down the monitor
Collecting data.
INFO[3726] Starting CollectData for all monitor tests   
INFO[3726]   Starting CollectData for [Monitor:watch-namespaces][Jira:"Test Framework"] monitor test watch-namespaces collection 
INFO[3726]   Finished CollectData for [Monitor:watch-namespaces][Jira:"Test Framework"] monitor test watch-namespaces collection 
INFO[3726] Finished CollectData for all monitor tests   
Computing intervals.
Evaluating tests.
Cleaning up.
INFO[3726] beginning cleanup                             monitorTest=watch-namespaces
Serializing results.
Writing to storage.
 m.startTime = 2025-09-26 11:31:07.177184 +0800 CST m=+226.458576459
 m.stopTime  = 2025-09-26 12:29:27.476377 +0800 CST m=+3726.772710917
Processing monitorTest: watch-namespaces
 finalIntervals size = 30
 first interval time: From = 2025-09-26 11:31:07.186684 +0800 CST m=+226.468076251; To = 2025-09-26 11:31:07.186684 +0800 CST m=+226.468076251
 last interval time: From = 2025-09-26 12:29:27.475089 +0800 CST m=+3726.771423001; To = 2025-09-26 12:29:27.475089 +0800 CST m=+3726.771423001
Writing junits.
Writing JUnit report to e2e-monitor-tests__20250926-032805.xml
14 pass, 0 flaky, 1 skip (1h1m22s)
INFO[3729] Successfully removed tmpDir /var/folders/wl/t8n9cyz511q96jx5xvjfwkn40000gn/T/openshift-tests1409893501 

Assisted-by: Claude Code

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@kuiwang02
Copy link
Contributor Author

/payload-aggregate periodic-ci-openshift-release-master-ci-4.21-e2e-gcp-ovn-techpreview 5

Copy link
Contributor

openshift-ci bot commented Sep 26, 2025

@kuiwang02: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-ci-4.21-e2e-gcp-ovn-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/62f09790-9a93-11f0-887b-65e2990bc61a-0

@kuiwang02
Copy link
Contributor Author

/test okd-scos-e2e-aws-ovn

@kuiwang02
Copy link
Contributor Author

/verified by @kuiwang02

@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Sep 28, 2025
@openshift-ci-robot
Copy link

@kuiwang02: This PR has been marked as verified by @kuiwang02.

In response to this:

/verified by @kuiwang02

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@jianzhangbjz
Copy link
Contributor

/approve
/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Sep 28, 2025
Copy link
Contributor

openshift-ci bot commented Sep 28, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jianzhangbjz, kuiwang02

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 28, 2025
@openshift-ci-robot
Copy link

/retest-required

Remaining retests: 0 against base HEAD b437870 and 2 for PR HEAD c36cbee in total

@kuiwang02
Copy link
Contributor Author

/retest-required

Copy link
Contributor

openshift-ci bot commented Sep 28, 2025

@kuiwang02: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/okd-scos-e2e-aws-ovn c36cbee link false /test okd-scos-e2e-aws-ovn

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@kuiwang02
Copy link
Contributor Author

/retest-required

@openshift-merge-bot openshift-merge-bot bot merged commit d839ba9 into openshift:main Sep 29, 2025
12 of 13 checks passed
@kuiwang02 kuiwang02 deleted the enhancefilters branch September 29, 2025 03:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. verified Signifies that the PR passed pre-merge verification criteria
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants