Skip to content

OPRUN-4181: UPSTREAM: <carry>: support singleown cases in disconnected #524

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Oct 17, 2025
Merged

OPRUN-4181: UPSTREAM: <carry>: support singleown cases in disconnected #524

merged 1 commit into from
Oct 17, 2025

Conversation

kuiwang02
Copy link
Contributor

@kuiwang02 kuiwang02 commented Oct 16, 2025 •
edited
Loading

Why / Problem Statement

OLMv1 singleown test cases were marked as [Skipped:Disconnected][Serial] which prevented them from:

  1. Running in disconnected/air-gapped environments
  2. Running in parallel with other tests

This limited test coverage in disconnected environments and increased overall test execution time.

What / Solution

This PR makes package names configurable in singleown testdata and enables execution with both connected and disconnected environments by:

  1. Adding in-cluster bundle/catalog support - Tests now build bundles and catalogs inside the cluster using embedded testdata (via go-bindata), enabling disconnected environment support
  2. Making package names configurable - Introduced {{ PACKAGE-NAME }} template variable throughout testdata files
  3. Making crd names configurable - Introduced {{ crdSuffix }} template variable throughout testdata files
  4. Enabling parallel execution - Removed [Skipped:Disconnected] tags to allow tests to run in disconnected environments with some of them in parallel

Changes Made

Testdata Templates

testdata/singleown/bundle/metadata/annotations.yaml

  • Added {{ PACKAGE-NAME }} template variable to operators.operatorframework.io.bundle.package.v1 annotation

testdata/singleown/bundle/manifests/webhook-operator.clusterserviceversion.yaml

  • Line 35: Changed name: {{ PACKAGE-NAME }}.v0.0.5 to name: "{{ PACKAGE-NAME }}.v0.0.5" (added quotes for YAML compliance)

testdata/singleown/index/configs/index.yaml

  • Replaced 7 occurrences of hardcoded "webhook-operator" with "{{ PACKAGE-NAME }}":
    • Package schema name (line 3)
    • Channel package reference (line 9)
    • Bundle name (line 18)
    • Bundle package reference (line 19)
    • Package property value (line 41)

New testdata files created:

  • Complete testdata/singleown/ directory structure including:
    • Bundle manifests (CSV, CRDs, RBAC resources)
    • Bundle metadata and Dockerfile
    • Index catalog configuration
    • Scorecard test configuration

Generated Bindata Files

pkg/bindata/singleown/bundle/bundle.go - Auto-generated file embedding bundle assets (461 lines added)

pkg/bindata/singleown/index/index.go - Auto-generated file embedding index catalog (271 lines added)

Test Implementation: test/olmv1-singleownnamespace.go

All Tests:

  • Removed [Skipped:Disconnected] tag to enable disconnected environment execution
  • Removed Serial tag from Tests 1, 2, 3, 5 to enable parallel execution
  • Changed from external registries (quay.io/olmtest, registry.redhat.io) to in-cluster image registry
  • Added helpers.RequireImageRegistry(ctx) check
  • Changed BeforeEach(func() to BeforeEach(func(ctx SpecContext) for context support

Build Configuration

Makefile

  • Added bindata generation targets for singleown testdata:
.PHONY: bindata/singleown/bundle
bindata/singleown/bundle:
    go-bindata -o pkg/bindata/singleown/bundle/bundle.go -pkg bundle -prefix testdata/singleown/bundle ./testdata/singleown/bundle/...

.PHONY: bindata/singleown/index
bindata/singleown/index:
    go-bindata -o pkg/bindata/singleown/index/index.go -pkg index -prefix testdata/singleown/index ./testdata/singleown/index/...

openshift_payload_olmv1.json

  • Updated test configuration to reference new test structure

Key Technical Decisions

  1. Template-based Package Name Configuration

    • Decision: Use {{ PACKAGE-NAME }} template variable in testdata, similar to existing {{ CRD-SUFFIX }} pattern
    • Rationale: Allows runtime generation of unique package names while keeping testdata files generic; follows established patterns in codebase; maintains separation between test logic and test data
    • Impact: Each test can now dynamically specify its own package name without modifying testdata files

  2. Random String vs Ginkgo Process ID for CRD Suffix

    • Rationale: More flexible and not dependent on Ginkgo's parallel execution mechanism; consistent with other resource naming patterns (namespace, SA, CRB, CE all use unique); avoids potential conflicts in different execution environments; random strings are more collision-resistant
    • Implementation: Tests 1, 2, 5 use crdSuffix = unique (reuse existing random string); Test 3 uses crdSuffix = rand.String(4) (no unique variable in that scope)

  3. Unique Package Name Patterns Per Test Suite

    • Decision: Each test suite uses a different naming pattern:
      • Test 1 (singleNamespace): singleown-operator-single-<random>
      • Test 2 (ownNamespace): singleown-operator-own-<random>
      • Test 3 (bothns): singleown-operator-both-<random>
      • Test 5 (invalid watch): singleown-operator-<random>
    • Rationale: Makes it easy to identify which test created which resources during debugging; helps with troubleshooting failed tests by matching resource names to test suites; provides clear traceability in cluster resource listings
    • Alternative Considered: Use same pattern for all tests - Rejected due to reduced debuggability

  4. In-cluster Bundle and Catalog for Disconnected Support

    • Decision: Build bundles and catalogs inside the cluster using embedded testdata (go-bindata) instead of using external registries
    • Rationale: Enables tests to run in disconnected/air-gapped environments; follows the same pattern as webhook tests; eliminates dependency on external image registries; bundles are built from embedded testdata at runtime
    • Implementation: Used helpers.NewCatalogAndClusterBundles() with singleown bindata assets
    • Impact: Tests can now run in disconnected environments, expanding test coverage

  5. Parallel Execution Enablement

    • Decision: Remove Serial tag from Tests 1, 2, 3, 5 to enable parallel execution
    • Rationale: With unique package names, tests no longer conflict; parallel execution significantly reduces total test time; follows best practices for Ginkgo tests
    • Exception: Test 4 remains Serial as it tests failure scenarios with webhook testdata

  6. Test 4 Scope Change

    • Decision: Test 4 was changed from using "openshift-pipelines-operator-rh" to using webhook testdata
    • Rationale: Test 4 is a negative test checking operator behavior when ownNamespace mode is not supported; webhook operator consistently supports only AllNamespaces mode, making it perfect for this test; using testdata instead of external registry enables disconnected support; webhook testdata is already available and embedded
    • Impact: Test 4 now uses in-cluster webhook bundle/catalog, runs in disconnected environments, but remains Serial since it's testing failure scenarios

Testing

INFO[0195] Found 0 must-gather tests                    
started: 0/1/5 "[sig-olmv1][OCPFeatureGate:NewOLMOwnSingleNamespace] OLMv1 operator installation should reject invalid watch namespace configuration and update the status conditions accordingly should fail to install the ClusterExtension when watch namespace is invalid"

started: 0/2/5 "[sig-olmv1][OCPFeatureGate:NewOLMOwnSingleNamespace] OLMv1 operator installation support for ownNamespace and single namespace watch mode with operator should install cluster extensions successfully in both watch modes"

started: 0/3/5 "[sig-olmv1][OCPFeatureGate:NewOLMOwnSingleNamespace] OLMv1 operator installation support for ownNamespace watch mode with operator should install a cluster extension successfully"

started: 0/4/5 "[sig-olmv1][OCPFeatureGate:NewOLMOwnSingleNamespace] OLMv1 operator installation support for singleNamespace watch mode with operator should install a cluster extension successfully"


passed: (42.8s) 2025-10-16T08:44:37 "[sig-olmv1][OCPFeatureGate:NewOLMOwnSingleNamespace] OLMv1 operator installation support for ownNamespace watch mode with operator should install a cluster extension successfully"


passed: (49.9s) 2025-10-16T08:44:44 "[sig-olmv1][OCPFeatureGate:NewOLMOwnSingleNamespace] OLMv1 operator installation support for singleNamespace watch mode with operator should install a cluster extension successfully"


passed: (51.4s) 2025-10-16T08:44:45 "[sig-olmv1][OCPFeatureGate:NewOLMOwnSingleNamespace] OLMv1 operator installation should reject invalid watch namespace configuration and update the status conditions accordingly should fail to install the ClusterExtension when watch namespace is invalid"


passed: (1m5s) 2025-10-16T08:44:59 "[sig-olmv1][OCPFeatureGate:NewOLMOwnSingleNamespace] OLMv1 operator installation support for ownNamespace and single namespace watch mode with operator should install cluster extensions successfully in both watch modes"

started: 0/5/5 "[sig-olmv1][OCPFeatureGate:NewOLMOwnSingleNamespace][Serial] OLMv1 operator installation support for ownNamespace watch mode with an operator that does not support ownNamespace installation mode should fail to install a cluster extension successfully"


passed: (37.8s) 2025-10-16T08:45:43 "[sig-olmv1][OCPFeatureGate:NewOLMOwnSingleNamespace][Serial] OLMv1 operator installation support for ownNamespace watch mode with an operator that does not support ownNamespace installation mode should fail to install a cluster extension successfully"

Shutting down the monitor
Collecting data.
INFO[0317] Starting CollectData for all monitor tests   
INFO[0317]   Starting CollectData for [Monitor:watch-namespaces][Jira:"Test Framework"] monitor test watch-namespaces collection 
INFO[0317]   Finished CollectData for [Monitor:watch-namespaces][Jira:"Test Framework"] monitor test watch-namespaces collection 
INFO[0317] Finished CollectData for all monitor tests   
Computing intervals.
Evaluating tests.
Cleaning up.
INFO[0317] beginning cleanup                             monitorTest=watch-namespaces
Serializing results.
Writing to storage.
m.startTime = 2025-10-16 16:43:41.569599 +0800 CST m=+195.256516251
m.stopTime  = 2025-10-16 16:45:43.420589 +0800 CST m=+317.109160710
Processing monitorTest: watch-namespaces
finalIntervals size = 10
first interval time: From = 2025-10-16 16:43:41.575319 +0800 CST m=+195.262236626; To = 2025-10-16 16:43:41.575319 +0800 CST m=+195.262236626
last interval time: From = 2025-10-16 16:45:43.420223 +0800 CST m=+317.108794501; To = 2025-10-16 16:45:43.420223 +0800 CST m=+317.108794501
Writing junits.
Writing JUnit report to e2e-monitor-tests__20251016-084039.xml
5 pass, 0 flaky, 0 skip (5m4s)

Template Variables in Use

  • {{ PACKAGE-NAME }} - New variable added in this PR for unique package names
  • {{ CRD-SUFFIX }} - Existing variable for unique CRD names
  • {{ TEST-BUNDLE }} - Existing variable for bundle image reference
  • {{ TEST-CONTROLLER }} - Existing variable for controller image reference
  • {{ NAMESPACE }} - Existing variable for namespace reference for buildconfig and image

Assisted-by: Claude Code

@openshift-ci-robot
Copy link

openshift-ci-robot commented Oct 16, 2025 •
edited by openshift-ci bot
Loading

@kuiwang02: This pull request references OPRUN-4181 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.21.0" version, but no target version was set.

In response to this:

WIP.....

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Oct 16, 2025
@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 16, 2025
@kuiwang02
Copy link
Contributor Author

/payload-aggregate periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-ovn-ipv6-techpreview 5

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Oct 16, 2025

@kuiwang02: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-ovn-ipv6-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/a5d6a670-aa57-11f0-9255-fd1d8f920f61-0

@kuiwang02
Copy link
Contributor Author

/payload-aggregate periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-serial-ovn-ipv6-techpreview-1of2 5

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Oct 16, 2025

@kuiwang02: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-serial-ovn-ipv6-techpreview-1of2

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/af9bb7e0-aa57-11f0-8bdf-dd4979e6d9f5-0

@kuiwang02
Copy link
Contributor Author

/payload-aggregate periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-serial-ovn-ipv6-techpreview-2of2 5

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Oct 16, 2025

@kuiwang02: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-serial-ovn-ipv6-techpreview-2of2

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/b5c30920-aa57-11f0-8353-342853163df4-0

@kuiwang02
Copy link
Contributor Author

/payload-aggregate periodic-ci-openshift-release-master-ci-4.21-e2e-gcp-ovn-techpreview 5

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Oct 16, 2025

@kuiwang02: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-ci-4.21-e2e-gcp-ovn-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/bdac55b0-aa57-11f0-869f-0cf0638b5b59-0

@kuiwang02
Copy link
Contributor Author

/payload-aggregate periodic-ci-openshift-release-master-ci-4.21-e2e-gcp-ovn-techpreview-serial 5

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Oct 16, 2025

@kuiwang02: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-ci-4.21-e2e-gcp-ovn-techpreview-serial

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/c52b6ec0-aa57-11f0-9ec1-3bca17589a75-0

@kuiwang02
Copy link
Contributor Author

/payload-aggregate periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-ovn-ipv6-techpreview 5

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Oct 16, 2025

@kuiwang02: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-ovn-ipv6-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/5134b010-aa6d-11f0-814e-fd629be74c60-0

@kuiwang02
Copy link
Contributor Author

/payload-aggregate periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-serial-ovn-ipv6-techpreview-1of2 5

@kuiwang02
Copy link
Contributor Author

/payload-aggregate periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-serial-ovn-ipv6-techpreview-2of2 5

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Oct 16, 2025

@kuiwang02: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-serial-ovn-ipv6-techpreview-1of2

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/58e3d9d0-aa6d-11f0-8be5-237e58fc54b2-0

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Oct 16, 2025

@kuiwang02: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-serial-ovn-ipv6-techpreview-2of2

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/5fcad190-aa6d-11f0-99cd-cc61929f009b-0

@kuiwang02
Copy link
Contributor Author

/payload-aggregate periodic-ci-openshift-release-master-ci-4.21-e2e-gcp-ovn-techpreview 5

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Oct 16, 2025

@kuiwang02: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-ci-4.21-e2e-gcp-ovn-techpreview

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/677c4540-aa6d-11f0-8ba2-dca1125a2d10-0

@kuiwang02
Copy link
Contributor Author

/payload-aggregate periodic-ci-openshift-release-master-ci-4.21-e2e-gcp-ovn-techpreview-serial 5

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Oct 16, 2025

@kuiwang02: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-ci-4.21-e2e-gcp-ovn-techpreview-serial

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/7516a1a0-aa6d-11f0-80c9-66446d50a317-0

@kuiwang02
Copy link
Contributor Author

/retest-required

@kuiwang02
Copy link
Contributor Author

/payload-aggregate periodic-ci-openshift-release-master-ci-4.21-e2e-gcp-ovn-techpreview-serial 5

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Oct 17, 2025

@kuiwang02: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-ci-4.21-e2e-gcp-ovn-techpreview-serial

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/6ffa1d50-aaf4-11f0-83a9-2b9e0bee43e0-0

@openshift-ci-robot
Copy link

openshift-ci-robot commented Oct 17, 2025 •
edited by openshift-ci bot
Loading

@kuiwang02: This pull request references OPRUN-4181 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.21.0" version, but no target version was set.

In response to this:

Why / Problem Statement

OLMv1 singleown test cases were marked as [Skipped:Disconnected][Serial] which prevented them from:

  1. Running in disconnected/air-gapped environments
  2. Running in parallel with other tests

This limited test coverage in disconnected environments and increased overall test execution time.

What / Solution

This PR makes package names configurable in singleown testdata and enables execution with both connected and disconnected environments by:

  1. Adding in-cluster bundle/catalog support - Tests now build bundles and catalogs inside the cluster using embedded testdata (via go-bindata), enabling disconnected environment support
  2. Making package names configurable - Introduced {{ PACKAGE-NAME }} template variable throughout testdata files
  3. Making crd names configurable - Introduced {{ crdSuffix }} template variable throughout testdata files
  4. Enabling parallel execution - Removed [Skipped:Disconnected] tags to allow tests to run in disconnected environments with some of them in parallel

Changes Made

Testdata Templates

testdata/singleown/bundle/metadata/annotations.yaml

  • Added {{ PACKAGE-NAME }} template variable to operators.operatorframework.io.bundle.package.v1 annotation

testdata/singleown/bundle/manifests/webhook-operator.clusterserviceversion.yaml

  • Line 35: Changed name: {{ PACKAGE-NAME }}.v0.0.5 to name: "{{ PACKAGE-NAME }}.v0.0.5" (added quotes for YAML compliance)

testdata/singleown/index/configs/index.yaml

  • Replaced 7 occurrences of hardcoded "webhook-operator" with "{{ PACKAGE-NAME }}":
  • Package schema name (line 3)
  • Channel package reference (line 9)
  • Bundle name (line 18)
  • Bundle package reference (line 19)
  • Package property value (line 41)

New testdata files created:

  • Complete testdata/singleown/ directory structure including:
  • Bundle manifests (CSV, CRDs, RBAC resources)
  • Bundle metadata and Dockerfile
  • Index catalog configuration
  • Scorecard test configuration

Generated Bindata Files

pkg/bindata/singleown/bundle/bundle.go - Auto-generated file embedding bundle assets (461 lines added)

pkg/bindata/singleown/index/index.go - Auto-generated file embedding index catalog (271 lines added)

Test Implementation: test/olmv1-singleownnamespace.go

All Tests:

  • Removed [Skipped:Disconnected] tag to enable disconnected environment execution
  • Removed Serial tag from Tests 1, 2, 3, 5 to enable parallel execution
  • Changed from external registries (quay.io/olmtest, registry.redhat.io) to in-cluster image registry
  • Added helpers.RequireImageRegistry(ctx) check
  • Changed BeforeEach(func() to BeforeEach(func(ctx SpecContext) for context support

Build Configuration

Makefile

  • Added bindata generation targets for singleown testdata:
.PHONY: bindata/singleown/bundle
bindata/singleown/bundle:
   go-bindata -o pkg/bindata/singleown/bundle/bundle.go -pkg bundle -prefix testdata/singleown/bundle ./testdata/singleown/bundle/...

.PHONY: bindata/singleown/index
bindata/singleown/index:
   go-bindata -o pkg/bindata/singleown/index/index.go -pkg index -prefix testdata/singleown/index ./testdata/singleown/index/...

openshift_payload_olmv1.json

  • Updated test configuration to reference new test structure

Key Technical Decisions

  1. Template-based Package Name Configuration
  • Decision: Use {{ PACKAGE-NAME }} template variable in testdata, similar to existing {{ CRD-SUFFIX }} pattern
  • Rationale: Allows runtime generation of unique package names while keeping testdata files generic; follows established patterns in codebase; maintains separation between test logic and test data
  • Impact: Each test can now dynamically specify its own package name without modifying testdata files
  1. Random String vs Ginkgo Process ID for CRD Suffix
  • Rationale: More flexible and not dependent on Ginkgo's parallel execution mechanism; consistent with other resource naming patterns (namespace, SA, CRB, CE all use unique); avoids potential conflicts in different execution environments; random strings are more collision-resistant
  • Implementation: Tests 1, 2, 5 use crdSuffix = unique (reuse existing random string); Test 3 uses crdSuffix = rand.String(4) (no unique variable in that scope)
  1. Unique Package Name Patterns Per Test Suite
  • Decision: Each test suite uses a different naming pattern:
    • Test 1 (singleNamespace): singleown-operator-single-<random>
    • Test 2 (ownNamespace): singleown-operator-own-<random>
    • Test 3 (bothns): singleown-operator-both-<random>
    • Test 5 (invalid watch): singleown-operator-<random>
  • Rationale: Makes it easy to identify which test created which resources during debugging; helps with troubleshooting failed tests by matching resource names to test suites; provides clear traceability in cluster resource listings
  • Alternative Considered: Use same pattern for all tests - Rejected due to reduced debuggability
  1. In-cluster Bundle and Catalog for Disconnected Support
  • Decision: Build bundles and catalogs inside the cluster using embedded testdata (go-bindata) instead of using external registries
  • Rationale: Enables tests to run in disconnected/air-gapped environments; follows the same pattern as webhook tests; eliminates dependency on external image registries; bundles are built from embedded testdata at runtime
  • Implementation: Used helpers.NewCatalogAndClusterBundles() with singleown bindata assets
  • Impact: Tests can now run in disconnected environments, expanding test coverage
  1. Parallel Execution Enablement
  • Decision: Remove Serial tag from Tests 1, 2, 3, 5 to enable parallel execution
  • Rationale: With unique package names, tests no longer conflict; parallel execution significantly reduces total test time; follows best practices for Ginkgo tests
  • Exception: Test 4 remains Serial as it tests failure scenarios with webhook testdata
  1. Test 4 Scope Change
  • Decision: Test 4 was changed from using "openshift-pipelines-operator-rh" to using webhook testdata
  • Rationale: Test 4 is a negative test checking operator behavior when ownNamespace mode is not supported; webhook operator consistently supports only AllNamespaces mode, making it perfect for this test; using testdata instead of external registry enables disconnected support; webhook testdata is already available and embedded
  • Impact: Test 4 now uses in-cluster webhook bundle/catalog, runs in disconnected environments, but remains Serial since it's testing failure scenarios

Testing

INFO[0195] Found 0 must-gather tests                    
started: 0/1/5 "[sig-olmv1][OCPFeatureGate:NewOLMOwnSingleNamespace] OLMv1 operator installation should reject invalid watch namespace configuration and update the status conditions accordingly should fail to install the ClusterExtension when watch namespace is invalid"

started: 0/2/5 "[sig-olmv1][OCPFeatureGate:NewOLMOwnSingleNamespace] OLMv1 operator installation support for ownNamespace and single namespace watch mode with operator should install cluster extensions successfully in both watch modes"

started: 0/3/5 "[sig-olmv1][OCPFeatureGate:NewOLMOwnSingleNamespace] OLMv1 operator installation support for ownNamespace watch mode with operator should install a cluster extension successfully"

started: 0/4/5 "[sig-olmv1][OCPFeatureGate:NewOLMOwnSingleNamespace] OLMv1 operator installation support for singleNamespace watch mode with operator should install a cluster extension successfully"


passed: (42.8s) 2025-10-16T08:44:37 "[sig-olmv1][OCPFeatureGate:NewOLMOwnSingleNamespace] OLMv1 operator installation support for ownNamespace watch mode with operator should install a cluster extension successfully"


passed: (49.9s) 2025-10-16T08:44:44 "[sig-olmv1][OCPFeatureGate:NewOLMOwnSingleNamespace] OLMv1 operator installation support for singleNamespace watch mode with operator should install a cluster extension successfully"


passed: (51.4s) 2025-10-16T08:44:45 "[sig-olmv1][OCPFeatureGate:NewOLMOwnSingleNamespace] OLMv1 operator installation should reject invalid watch namespace configuration and update the status conditions accordingly should fail to install the ClusterExtension when watch namespace is invalid"


passed: (1m5s) 2025-10-16T08:44:59 "[sig-olmv1][OCPFeatureGate:NewOLMOwnSingleNamespace] OLMv1 operator installation support for ownNamespace and single namespace watch mode with operator should install cluster extensions successfully in both watch modes"

started: 0/5/5 "[sig-olmv1][OCPFeatureGate:NewOLMOwnSingleNamespace][Serial] OLMv1 operator installation support for ownNamespace watch mode with an operator that does not support ownNamespace installation mode should fail to install a cluster extension successfully"


passed: (37.8s) 2025-10-16T08:45:43 "[sig-olmv1][OCPFeatureGate:NewOLMOwnSingleNamespace][Serial] OLMv1 operator installation support for ownNamespace watch mode with an operator that does not support ownNamespace installation mode should fail to install a cluster extension successfully"

Shutting down the monitor
Collecting data.
INFO[0317] Starting CollectData for all monitor tests   
INFO[0317]   Starting CollectData for [Monitor:watch-namespaces][Jira:"Test Framework"] monitor test watch-namespaces collection 
INFO[0317]   Finished CollectData for [Monitor:watch-namespaces][Jira:"Test Framework"] monitor test watch-namespaces collection 
INFO[0317] Finished CollectData for all monitor tests   
Computing intervals.
Evaluating tests.
Cleaning up.
INFO[0317] beginning cleanup                             monitorTest=watch-namespaces
Serializing results.
Writing to storage.
m.startTime = 2025-10-16 16:43:41.569599 +0800 CST m=+195.256516251
m.stopTime  = 2025-10-16 16:45:43.420589 +0800 CST m=+317.109160710
Processing monitorTest: watch-namespaces
finalIntervals size = 10
first interval time: From = 2025-10-16 16:43:41.575319 +0800 CST m=+195.262236626; To = 2025-10-16 16:43:41.575319 +0800 CST m=+195.262236626
last interval time: From = 2025-10-16 16:45:43.420223 +0800 CST m=+317.108794501; To = 2025-10-16 16:45:43.420223 +0800 CST m=+317.108794501
Writing junits.
Writing JUnit report to e2e-monitor-tests__20251016-084039.xml
5 pass, 0 flaky, 0 skip (5m4s)

Template Variables in Use

  • {{ PACKAGE-NAME }} - New variable added in this PR for unique package names
  • {{ CRD-SUFFIX }} - Existing variable for unique CRD names
  • {{ TEST-BUNDLE }} - Existing variable for bundle image reference
  • {{ TEST-CONTROLLER }} - Existing variable for controller image reference
  • {{ NAMESPACE }} - Existing variable for namespace reference for buildconfig and image

Assisted-by: Claude Code

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@kuiwang02
Copy link
Contributor Author

kuiwang02 commented Oct 17, 2025 •
edited by openshift-ci bot
Loading

/retitle OPRUN-4181: UPSTREAM: : support singleown cases in disconnected

@openshift-ci openshift-ci bot changed the title [WIP]OPRUN-4181: UPSTREAM: <carry>: support singleown cases in disconnected OPRUN-4181: UPSTREAM: <carry>: support singleown cases in disconnected Oct 17, 2025
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 17, 2025
@kuiwang02
Copy link
Contributor Author

SingleOwn Test Results Summary (Connected & Disconnected Environments)

SingleOwn Parallel Case in Disconnected Environment

Command:
/payload-aggregate periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-ovn-ipv6-techpreview 5

Run Reference:
Run Dashboard

Prow Jobs:
Job 1 Job 2 Job 3 Job 4 Job 5

Example log output:

passed: (50.3s) 2025-10-16T12:28:59 "[sig-olmv1][OCPFeatureGate:NewOLMOwnSingleNamespace] OLMv1 operator installation support for singleNamespace watch mode with operator should install a cluster extension successfully"
passed: (49.1s) 2025-10-16T12:29:34 "[sig-olmv1][OCPFeatureGate:NewOLMOwnSingleNamespace] OLMv1 operator installation should reject invalid watch namespace configuration and update the status conditions accordingly should fail to install the ClusterExtension when watch namespace is invalid"
passed: (44.4s) 2025-10-16T12:30:12 "[sig-olmv1][OCPFeatureGate:NewOLMOwnSingleNamespace] OLMv1 operator installation support for ownNamespace watch mode with operator should install a cluster extension successfully"
passed: (59.9s) 2025-10-16T12:30:37 "[sig-olmv1][OCPFeatureGate:NewOLMOwnSingleNamespace] OLMv1 operator installation support for ownNamespace and single namespace watch mode with operator should install cluster extensions successfully in both watch modes"

✅ Conclusion:
The parallel SingleOwn test case passed in the disconnected environment.

SingleOwn Serial Cases in Disconnected Environment

Command:
/payload-aggregate periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-serial-ovn-ipv6-techpreview-2of2 5

Run Reference:
Run Dashboard

Prow Jobs:

Job 1 Job 2 Job 3 Job 4 Job 5

Example log outputs:

passed: (25.3s) 2025-10-16T12:35:10 "[sig-olmv1][OCPFeatureGate:NewOLMOwnSingleNamespace][Serial] OLMv1 operator installation support for ownNamespace watch mode with an operator that does not support ownNamespace installation mode should fail to install a cluster extension successfully"

✅ Conclusion:
All serial SingleOwn test cases passed in the disconnected environment.

SingleOwn Parallel Case in Connected Environment

Command:
/payload-aggregate periodic-ci-openshift-release-master-ci-4.21-e2e-gcp-ovn-techpreview 5

Run Reference:
Run Dashboard

Prow Jobs:

Job 1 Job 2 Job 3 Job 4 Job 5

Example log output:

passed: (58.6s) 2025-10-16T11:01:34 "[sig-olmv1][OCPFeatureGate:NewOLMOwnSingleNamespace] OLMv1 operator installation support for singleNamespace watch mode with operator should install a cluster extension successfully"
passed: (54.4s) 2025-10-16T11:02:50 "[sig-olmv1][OCPFeatureGate:NewOLMOwnSingleNamespace] OLMv1 operator installation support for ownNamespace watch mode with operator should install a cluster extension successfully"
passed: (55.3s) 2025-10-16T11:04:14 "[sig-olmv1][OCPFeatureGate:NewOLMOwnSingleNamespace] OLMv1 operator installation should reject invalid watch namespace configuration and update the status conditions accordingly should fail to install the ClusterExtension when watch namespace is invalid"
passed: (2m34s) 2025-10-16T11:04:15 "[sig-olmv1][OCPFeatureGate:NewOLMOwnSingleNamespace] OLMv1 operator installation support for ownNamespace and single namespace watch mode with operator should install cluster extensions successfully in both watch modes"

✅ Conclusion:
The parallel SingleOwn test case passed in the connected environment.

SingleOwn Serial Cases in Connected Environment

Command:
/payload-aggregate periodic-ci-openshift-release-master-ci-4.21-e2e-gcp-ovn-techpreview-serial 5

Run Reference:
Run Dashboard

Prow Jobs:

Job 1 Job 2 Job 3 Job 4 Job 5

Example log outputs:

passed: (31.9s) 2025-10-16T13:33:45 "[sig-olmv1][OCPFeatureGate:NewOLMOwnSingleNamespace][Serial] OLMv1 operator installation support for ownNamespace watch mode with an operator that does not support ownNamespace installation mode should fail to install a cluster extension successfully"

✅ Conclusion:
All serial SingleOwn test cases passed in the connected environment.

Overall Summary

All SingleOwn test cases — both Parallel and Serial, in Connected and Disconnected environments — have passed successfully. This confirms that the updated test logic and image initialization now work reliably across both network conditions.

@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Oct 17, 2025

@kuiwang02: trigger 4 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

  • periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-ovn-ipv6-techpreview
  • periodic-ci-openshift-release-master-nightly-4.21-e2e-metal-ipi-serial-ovn-ipv6-techpreview-2of2
  • periodic-ci-openshift-release-master-ci-4.21-e2e-gcp-ovn-techpreview
  • periodic-ci-openshift-release-master-ci-4.21-e2e-gcp-ovn-techpreview-serial

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/24a01e50-aaf8-11f0-8bfc-e91f98ae9557-0

@kuiwang02
Copy link
Contributor Author

/verified by @kuiwang02

@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Oct 17, 2025
@openshift-ci-robot
Copy link

@kuiwang02: This PR has been marked as verified by @kuiwang02.

In response to this:

/verified by @kuiwang02

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@jianzhangbjz
Copy link
Contributor

/approve
/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Oct 17, 2025
@openshift-ci OpenShift CI
Copy link
Contributor

openshift-ci bot commented Oct 17, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jianzhangbjz, kuiwang02

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 17, 2025
@openshift-merge-bot openshift-merge-bot bot merged commit e77207b into openshift:main Oct 17, 2025
11 of 12 checks passed
@kuiwang02 kuiwang02 deleted the supportsodisc branch October 17, 2025 05:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ankitathomas ankitathomas Awaiting requested review from ankitathomas

@dtfranz dtfranz Awaiting requested review from dtfranz

Assignees

@jianzhangbjz jianzhangbjz

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. verified Signifies that the PR passed pre-merge verification criteria

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@kuiwang02 @openshift-ci-robot @jianzhangbjz