Skip to content

[release-4.20] OCPBUGS-85155, OCPBUGS-87207: Enable Gateway API tests on vSphere and baremetal#31262

Open
gcs278 wants to merge 30 commits into
openshift:release-4.20from
gcs278:backport-gateway-tests-4.20
Open

[release-4.20] OCPBUGS-85155, OCPBUGS-87207: Enable Gateway API tests on vSphere and baremetal#31262
gcs278 wants to merge 30 commits into
openshift:release-4.20from
gcs278:backport-gateway-tests-4.20

Conversation

@gcs278
Copy link
Copy Markdown
Contributor

@gcs278 gcs278 commented Jun 5, 2026

Backport Gateway API test improvements to enable running on vSphere and baremetal platforms.

This is the 4.20 equivalent of #31139 (release-4.21).

Backported PRs

  1. NE-2090: adding one GIE automation test #30189 (Dependency)
  2. OCPBUGS-56281: gatewayapicontroller: Clean up resources when done #29900 (Dependency)
  3. OCPBUGS-56281: gatewayapicontroller: Use dynamic client for OLM #30397 (Dependency)
  4. NE-2056: auto negative tests of gatewayapicontroller #30599 (Dependency)
  5. NE-2292: Skip OLM tests when GatewayAPIWithoutOLM enabled #30896 (Dependency)
  6. NE-2292: Tests for gatewayAPIWithoutOLM featuregate graduation #30905 (Dependency)
  7. NE-2422: Skip Gateway API tests on dual-stack clusters #30942 (Dependency)
  8. NE-2520: Enable Gateway API tests on vSphere and baremetal #30946 (Target)

Why

The primary goal is to backport #30946 which enables Gateway API e2e tests to run on vSphere and baremetal platforms by gracefully handling missing LoadBalancer and DNS capabilities. PRs #30189 through #30942 are prerequisites that #30946 builds on — they restructure the test file, add constants and error handling, introduce GIE and negative tests, switch to dynamic client for OLM, and add feature gate handling that #30946 depends on. Including the full chain ensures clean cherry-picks and avoids future merge conflicts if any of these changes need to be backported independently later.

The GatewayAPIWithoutOLM feature gate tests are included but will not run — isNoOLMFeatureGateEnabled returns false early because the feature gate is not defined in openshift/api for this release branch. Without this early return, the feature gate lookup treats an undefined gate as a GA default and returns true, which would run the No OLM tests against a cluster using OLM-based provisioning and fail.

New tests

Will run (gated on GatewayAPIController)

  1. Ensure GIE is enabled after creating an inferencePool CRD (NE-2090: adding one GIE automation test #30189) — CIO GIE logic already on release-4.20 (openshift/cluster-ingress-operator@35c0143b7), this adds missing test coverage.
  2. Ensure istiod deployment and the istio could be deleted and then get recreated [Serial] (NE-2520: Enable Gateway API tests on vSphere and baremetal #30946) — Tests existing CIO reconciliation logic.
  3. Ensure gateway LB service and dnsrecords could be deleted and then get recreated [Serial] (NE-2520: Enable Gateway API tests on vSphere and baremetal #30946) — Tests existing CIO reconciliation logic. Part of the target PR for this backport.

Won't run (gated on GatewayAPIWithoutOLM — not defined in 4.20)

Five noOLM tests from #30599, #30896, #30905 — dead code until the openshift/api backport lands. Included because they're part of the cherry-pick chain.

Conflicts and fixup

All cherry-picks applied cleanly to the test files — no manual conflict resolution was needed in any .go test code.

Two commits had conflicts in vendor/modules.txt due to differing openshift/kubernetes fork hashes between release-4.20 and master. These were resolved by keeping the 4.20 hashes.

PR #30397 introduced transitive go.mod dependency bumps (docker v27.1.2→v27.3.1, fsnotify v1.7.0→v1.8.0, controller-runtime v0.19.0→v0.19.4, etc.) from a temporary operator-lifecycle-manager import that was later removed in the same PR. Since the test code only uses packages already in 4.20's vendor tree, the fixup commit reverts go.mod/go.sum to the exact release-4.20 originals, re-vendors, and regenerates zz_generated.annotations.go and test-reporting.yaml.

Dependency

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jun 5, 2026
@openshift-ci-robot
Copy link
Copy Markdown

openshift-ci-robot commented Jun 5, 2026

@gcs278: This pull request references NE-2286 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the epic to target either version "4.20." or "openshift-4.20.", but it targets "openshift-4.22" instead.

Details

In response to this:

Summary

  • Cherry-picks 8 PRs from master that bring Gateway API E2E tests up to date on release-4.20
  • Enables Gateway API tests on vSphere and baremetal platforms
  • Adds noOLM (GatewayAPIWithoutOLM) feature gate handling to tests
  • Adds negative tests, GIE test, dual-stack skip, and test reliability improvements

Cherry-picked PRs (in order)

  1. OCPBUGS-56281: gatewayapicontroller: Clean up resources when done #29900 — Major test restructure: cleanup, constants, timeouts, error handling
  2. NE-2090: adding one GIE automation test #30189 — GIE (Gateway Inference Extension) test
  3. OCPBUGS-56281: gatewayapicontroller: Use dynamic client for OLM #30397 — Use dynamic client for OLM (removes operator-framework vendor dependency)
  4. NE-2056: auto negative tests of gatewayapicontroller #30599 — Negative tests for gateway API controller
  5. NE-2292: Skip OLM tests when GatewayAPIWithoutOLM enabled #30896 — Skip OLM tests when GatewayAPIWithoutOLM feature gate is enabled
  6. NE-2292: Tests for gatewayAPIWithoutOLM featuregate graduation #30905 — Origin tests implementation for GatewayAPIWithoutOLM feature gate
  7. NE-2422: Skip Gateway API tests on dual-stack clusters #30942 — Skip Gateway API tests on dual-stack clusters
  8. NE-2520: Enable Gateway API tests on vSphere and baremetal #30946 — Enable Gateway API tests on vSphere and baremetal

Conflict Resolutions

Test plan

  • Run openshift-tests run-test for [sig-network-edge][OCPFeatureGate:GatewayAPIController] suite
  • Verify OLM-based tests still pass on clusters with OLM
  • Verify noOLM tests pass when GatewayAPIWithoutOLM feature gate is enabled

🤖 Generated with Claude Code

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Jun 5, 2026

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: b72adbc0-f195-4378-9e2a-f3af69e671a7

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci openshift-ci Bot requested review from deads2k and ffromani June 5, 2026 14:39
@openshift-ci openshift-ci Bot added the vendor-update Touching vendor dir or related files label Jun 5, 2026
@gcs278 gcs278 changed the title NE-2286: Backport Gateway API test improvements and noOLM support OCPBUGS-85155: Backport Gateway API E2E tests to release-4.20 Jun 5, 2026
@openshift-ci-robot openshift-ci-robot added jira/severity-low Referenced Jira bug's severity is low for the branch this PR is targeting. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. labels Jun 5, 2026
@openshift-ci-robot
Copy link
Copy Markdown

@gcs278: This pull request references Jira Issue OCPBUGS-85155, which is valid.

7 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.20.z) matches configured target version for branch (4.20.z)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)
  • release note type set to "Release Note Not Required"
  • dependent bug Jira Issue OCPBUGS-85149 is in the state Verified, which is one of the valid states (VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA))
  • dependent Jira Issue OCPBUGS-85149 targets the "4.21.z" version, which is one of the valid target versions: 4.21.0, 4.21.z
  • bug has dependents

Requesting review from QA contact:
/cc @melvinjoseph86

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Summary

Cherry-picks 8 PRs from master to bring Gateway API E2E tests up to date on release-4.20, enabling test coverage for vSphere, baremetal, dual-stack skip, negative tests, GIE, and the GatewayAPIWithoutOLM feature gate.

Cherry-picked PRs

PR Description
#29900 Test restructure: cleanup, constants, timeouts, error handling
#30189 GIE (Gateway Inference Extension) test
#30397 Use dynamic client for OLM (removes operator-framework vendor dep)
#30599 Negative tests for gateway API controller
#30896 Skip OLM tests when GatewayAPIWithoutOLM enabled (dependency for #30905)
#30905 GatewayAPIWithoutOLM test implementation (dependency for #30946)
#30942 Skip Gateway API tests on dual-stack clusters
#30946 Enable Gateway API tests on vSphere and baremetal

Conflict Resolutions

Test plan

  • Run openshift-tests run-test for [sig-network-edge][OCPFeatureGate:GatewayAPIController] suite
  • Verify OLM-based tests still pass on clusters with OLM

🤖 Generated with Claude Code

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci Bot requested a review from melvinjoseph86 June 5, 2026 14:40
@gcs278 gcs278 changed the title OCPBUGS-85155: Backport Gateway API E2E tests to release-4.20 [release-4.20] OCPBUGS-85155: Backport Gateway API E2E tests to release-4.20 Jun 5, 2026
@gcs278 gcs278 changed the title [release-4.20] OCPBUGS-85155: Backport Gateway API E2E tests to release-4.20 [release-4.20] OCPBUGS-85155: Enable Gateway API tests on vSphere and baremetal Jun 5, 2026
rhamini3 and others added 17 commits June 5, 2026 11:03
Check whether the slice of parent resource references in an httproute's
status is empty before indexing the slice.

Before this commit, the "Ensure HTTPRoute object is created" test
sometimes panicked with "runtime error: index out of range [0] with
length 0".

Similarly, check whether the slice of load-balancer ingress points in a
service's status is empty before indexing it.

* test/extended/router/gatewayapicontroller.go (buildGateway)
(createHttpRoute): Add checks.
Delete the gatewayclass and uninstall OSSM after all the Gateway API
controller tests are done.

Before this commit, the Gateway API controller tests left OSSM
installed, including the subscription, CSV, installplan, bundled CRDs,
RBAC resources, deployment, service, serviceaccount, etc., when the
tests were finished.  This clutter could cause problems for other tests,
or for the same test if it was run again.

The new cleanup logic uses the OperatorsV1 client from
github.com/operator-framework/operator-lifecycle-manager.  Importing
this package requires a replace stanza for openshift/api in go.mod.

    go get github.com/operator-framework/operator-lifecycle-manager@1b6752ec65faf4102f61121ebfbfb260c6d097f6
    go mod edit --replace=github.com/openshift/api=github.com/openshift/api@v0.0.0-20250710004639-926605d3338b
    go mod tidy
    go mod vendor

This vendors github.com/operator-framework/operator-lifecycle-manager
v0.30.1-0.20250114164243-1b6752ec65fa rather than the newest revision
in order to avoid bringing in additional problematic vendor bumps that
the newest revision would bring in.

This commit fixes OCPBUGS-56281.

https://issues.redhat.com/browse/OCPBUGS-56281

* go.mod: Vendor the operatorsv1 client code from
github.com/operator-framework/operator-lifecycle-manager.
* go.sum: Regenerate.
* test/extended/router/gatewayapicontroller.go: Delete the gatewayclass
that the test creates.  Use the OperatorsV1 client to look up the
Operator object for OSSM, and delete all the resources that the Operator
object references.
* vendor/*: Regenerate.
* test/extended/router/gatewayapicontroller.go: Add the error value to
some log messages that were missing it.
Log errors and then retry in the polling loop for the Subscription CR.

Before this commit, the gatewayapicontroller tests sometimes failed
because OSSM was still installing when this polling loop ran, and the
polling loop would fail on a "not found" error if the CR had not yet
been created.  In order to make the tests more reliable, they need to
retry on these errors.  For consistency with other polling loops, this
commit makes the polling loop retry on all errors (not just "not found"
errors).

* test/extended/router/gatewayapicontroller.go: Retry when the test
fails to get the OSSM subscription CR.
* test/extended/router/gatewayapicontroller.go: Increase the timeouts on
some polling loops that have been observed to flake but then succeed on
retry.
* test/extended/router/gatewayapicontroller.go (ingressNamespace): New
const.
(waitForIstioHealthy, createAndCheckGateway)
(assertGatewayLoadbalancerReady, assertDNSRecordStatus, createHttpRoute)
(assertHttpRouteSuccessful): Use the new const instead of function-level
variables or string literals.
* test/extended/router/gatewayapicontroller.go: Omit the namespace
when getting the Istio CR, which is cluster-scoped.
* test/extended/router/gatewayapicontroller.go (istioName): Declare
const.
(waitForIstioHealthy): Use the new const instead of a string literal.
* test/extended/router/gatewayapicontroller.go: Delete the Istio CR and
wait for the istiod pod to be deleted as part of the test cleanup.
Replace BeforeAll and AfterAll with BeforeEach and AfterEach,
respectively, and add explicit logic to prevent the cleanup logic
in AfterEach from running before the tests are all done.

Because of the way that Origin tests are run, BeforeAll and AfterAll
actually behave as BeforeEach and AfterEach.  That is, an AfterAll block
is run not once after all the specs in the Describe container are done
but rather once after each spec, and similarly, a BeforeAll block is run
before each spec.  Before this change, the cleanup logic was happening
after every spec, meaning OSSM was being uninstalled and re-installed
between each test, which caused a lot of churn and flakiness.

This change replaces BeforeAll and AfterAll because using them is
misleading.  This change also makes the tests faster and more reliable.

* gatewayapicontroller.go (gatewayClassName): Move declaration from the
Describe block to the package level so that helpers can use it.
(ossmAndOLMResourcesCreated, defaultGatewayclassAccepted)
(customGatewayclassAccepted, lbAndServiceAndDnsrecordAreCreated)
(httprouteObjectCreated, gieEnabled): New consts used to identify tests.
(testNames): New variable with a list of all tests in the file.
(Describe): Replace BeforeAll and AfterAll with BeforeEach and
AfterEach.  Use checkAllTestsDone in the AfterEach block to skip cleanup
while tests are still running.  Use a deferred call to markTestDone in
each test's It block to mark the test as done when it finishes.
(annotationKeyForTest): New helper.  Return the key for an annotation on
the default gatewayclass that indicates whether a given test is done.
(markTestDone): New helper.  Add an annotation to the default
gatewayclass to indicate that a test is done.
(checkAllTestsDone): New helper. Check the default gatewayclass's
annotations and return a Boolean value indicating whether all tests are
done.
* test/extended/pods/priorityclasses.go (excludedPriorityClassPods): Add
an override for Istio gateway pods.
* test/extended/cpu_partitioning/pods.go
(excludedBestEffortDeployments): Add an override for Istio gateway pods.
Instead of using the OperatorsV1 client from the
github.com/operator-framework/operator-lifecycle-manager package, use
the dynamic client, and drop the import for this package.  Vendoring
this package necessitated adding a replace stanza for openshift/api in
go.mod; this stanza can now be dropped.

The replace stanza was causing problems to people who wanted to import
openshift/origin to use the OpenShift Tests Extensions framework.

Follow-up to commit 407d63b.

* go.mod: Drop github.com/operator-framework/operator-lifecycle-manager,
and drop the replace rule for openshift/api.
* go.sum: Regenerate.
* test/extended/router/gatewayapicontroller.go: Use the dynamic client.
* vendor/*: Regenerate.
Log each deleted object during post-test cleanup.

* test/extended/router/gatewayapicontroller.go: Add logging.
Gateway API tests fail on IPv6 and dual-stack clusters, particularly on
baremetal platforms where catalog sources are typically disabled. This
prevents OSSM operator installation via OLM.

Replace AWS-specific dual-stack check with platform-agnostic detection
that checks the cluster's ServiceNetwork CIDRs for IPv6 addressing.

This will skip Gateway API tests on:
- Baremetal/vSphere/EquinixMetal IPv6 or dual-stack clusters
- AWS dual-stack clusters
- Any other platform with IPv6 networking

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@gcs278 gcs278 force-pushed the backport-gateway-tests-4.20 branch from 570bfac to a02fd86 Compare June 5, 2026 15:05
@gcs278
Copy link
Copy Markdown
Contributor Author

gcs278 commented Jun 5, 2026

This is also backporting https://redhat.atlassian.net/browse/OCPBUGS-56281 as a dependency, so I'm going to properly link it

/jira cherrypick OCPBUGS-56281

@openshift-ci-robot
Copy link
Copy Markdown

@gcs278: Jira Issue OCPBUGS-56281 has been cloned as Jira Issue OCPBUGS-87207. Will retitle bug to link to clone.
/retitle OCPBUGS-87207: [release-4.20] OCPBUGS-85155: Enable Gateway API tests on vSphere and baremetal

Details

In response to this:

This is also backporting https://redhat.atlassian.net/browse/OCPBUGS-56281 as a dependency, so I'm going to properly link it

/jira cherrypick OCPBUGS-56281

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci Bot changed the title [release-4.20] OCPBUGS-85155: Enable Gateway API tests on vSphere and baremetal OCPBUGS-87207: [release-4.20] OCPBUGS-85155: Enable Gateway API tests on vSphere and baremetal Jun 5, 2026
@openshift-ci-robot openshift-ci-robot added jira/severity-critical Referenced Jira bug's severity is critical for the branch this PR is targeting. and removed jira/severity-low Referenced Jira bug's severity is low for the branch this PR is targeting. labels Jun 5, 2026
@openshift-ci-robot
Copy link
Copy Markdown

@gcs278: This pull request references Jira Issue OCPBUGS-87207, which is valid. The bug has been moved to the POST state.

7 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.20.z) matches configured target version for branch (4.20.z)
  • bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)
  • release note text is set and does not match the template
  • dependent bug Jira Issue OCPBUGS-56281 is in the state Closed (Done), which is one of the valid states (VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA))
  • dependent Jira Issue OCPBUGS-56281 targets the "4.21.0" version, which is one of the valid target versions: 4.21.0, 4.21.z
  • bug has dependents

No GitHub users were found matching the public email listed for the QA contact in Jira (iamin@redhat.com), skipping review request.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Backport Gateway API test improvements to enable running on vSphere and baremetal platforms.

Backported PRs

  1. NE-2090: adding one GIE automation test #30189 (Dependency)
  2. OCPBUGS-56281: gatewayapicontroller: Clean up resources when done #29900 (Dependency)
  3. OCPBUGS-56281: gatewayapicontroller: Use dynamic client for OLM #30397 (Dependency)
  4. NE-2056: auto negative tests of gatewayapicontroller #30599 (Dependency)
  5. NE-2292: Skip OLM tests when GatewayAPIWithoutOLM enabled #30896 (Dependency)
  6. NE-2292: Tests for gatewayAPIWithoutOLM featuregate graduation #30905 (Dependency)
  7. NE-2422: Skip Gateway API tests on dual-stack clusters #30942 (Dependency)
  8. NE-2520: Enable Gateway API tests on vSphere and baremetal #30946 (Target)

Why

The primary goal is to backport #30946 which enables Gateway API e2e tests to run on vSphere and baremetal platforms by gracefully handling missing LoadBalancer and DNS capabilities. PRs #30189 through #30942 are prerequisites that #30946 builds on — they restructure the test file, add constants and error handling, introduce GIE and negative tests, switch to dynamic client for OLM, and add feature gate handling that #30946 depends on. Including the full chain ensures clean cherry-picks and avoids future merge conflicts if any of these changes need to be backported independently later.

The GatewayAPIWithoutOLM feature gate tests are included but will not run — isNoOLMFeatureGateEnabled returns false early because the feature gate is not defined in openshift/api for this release branch. Without this early return, the feature gate lookup treats an undefined gate as a GA default and returns true, which would run the No OLM tests against a cluster using OLM-based provisioning and fail.

Dependency

  • openshift/api backport of GatewayAPIWithoutOLM feature gate (disabled) to release-4.20. Once merged, the early return workaround in isNoOLMFeatureGateEnabled can be removed and the feature gate will resolve correctly as disabled.

All cherry-picks applied cleanly (only vendor/modules.txt needed ours-resolution for k8s fork hashes).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@gcs278 gcs278 changed the title OCPBUGS-87207: [release-4.20] OCPBUGS-85155: Enable Gateway API tests on vSphere and baremetal [release-4.20] OCPBUGS-85155, OCPBUGS-87207: Enable Gateway API tests on vSphere and baremetal Jun 5, 2026
@openshift-ci-robot
Copy link
Copy Markdown

@gcs278: This pull request references Jira Issue OCPBUGS-85155, which is valid.

7 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.20.z) matches configured target version for branch (4.20.z)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)
  • release note type set to "Release Note Not Required"
  • dependent bug Jira Issue OCPBUGS-85149 is in the state Verified, which is one of the valid states (VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA))
  • dependent Jira Issue OCPBUGS-85149 targets the "4.21.z" version, which is one of the valid target versions: 4.21.0, 4.21.z
  • bug has dependents

Requesting review from QA contact:
/cc @melvinjoseph86

The bug has been updated to refer to the pull request using the external bug tracker.

This pull request references Jira Issue OCPBUGS-87207, which is valid.

7 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.20.z) matches configured target version for branch (4.20.z)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)
  • release note text is set and does not match the template
  • dependent bug Jira Issue OCPBUGS-56281 is in the state Closed (Done), which is one of the valid states (VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA))
  • dependent Jira Issue OCPBUGS-56281 targets the "4.21.0" version, which is one of the valid target versions: 4.21.0, 4.21.z
  • bug has dependents

No GitHub users were found matching the public email listed for the QA contact in Jira (iamin@redhat.com), skipping review request.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Backport Gateway API test improvements to enable running on vSphere and baremetal platforms.

Backported PRs

  1. NE-2090: adding one GIE automation test #30189 (Dependency)
  2. OCPBUGS-56281: gatewayapicontroller: Clean up resources when done #29900 (Dependency)
  3. OCPBUGS-56281: gatewayapicontroller: Use dynamic client for OLM #30397 (Dependency)
  4. NE-2056: auto negative tests of gatewayapicontroller #30599 (Dependency)
  5. NE-2292: Skip OLM tests when GatewayAPIWithoutOLM enabled #30896 (Dependency)
  6. NE-2292: Tests for gatewayAPIWithoutOLM featuregate graduation #30905 (Dependency)
  7. NE-2422: Skip Gateway API tests on dual-stack clusters #30942 (Dependency)
  8. NE-2520: Enable Gateway API tests on vSphere and baremetal #30946 (Target)

Why

The primary goal is to backport #30946 which enables Gateway API e2e tests to run on vSphere and baremetal platforms by gracefully handling missing LoadBalancer and DNS capabilities. PRs #30189 through #30942 are prerequisites that #30946 builds on — they restructure the test file, add constants and error handling, introduce GIE and negative tests, switch to dynamic client for OLM, and add feature gate handling that #30946 depends on. Including the full chain ensures clean cherry-picks and avoids future merge conflicts if any of these changes need to be backported independently later.

The GatewayAPIWithoutOLM feature gate tests are included but will not run — isNoOLMFeatureGateEnabled returns false early because the feature gate is not defined in openshift/api for this release branch. Without this early return, the feature gate lookup treats an undefined gate as a GA default and returns true, which would run the No OLM tests against a cluster using OLM-based provisioning and fail.

Dependency

  • openshift/api backport of GatewayAPIWithoutOLM feature gate (disabled) to release-4.20. Once merged, the early return workaround in isNoOLMFeatureGateEnabled can be removed and the feature gate will resolve correctly as disabled.

All cherry-picks applied cleanly (only vendor/modules.txt needed ours-resolution for k8s fork hashes).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@gcs278 gcs278 force-pushed the backport-gateway-tests-4.20 branch from 5917fc4 to 94aaec5 Compare June 5, 2026 17:30
Reverts transitive dependency bumps from cherry-picks, re-vendors,
and regenerates zz_generated.annotations.go and test-reporting.yaml.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@gcs278 gcs278 force-pushed the backport-gateway-tests-4.20 branch from 94aaec5 to a76f868 Compare June 5, 2026 17:37
@openshift-ci-robot
Copy link
Copy Markdown

@gcs278: This pull request references Jira Issue OCPBUGS-85155, which is valid.

7 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.20.z) matches configured target version for branch (4.20.z)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)
  • release note type set to "Release Note Not Required"
  • dependent bug Jira Issue OCPBUGS-85149 is in the state Verified, which is one of the valid states (VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA))
  • dependent Jira Issue OCPBUGS-85149 targets the "4.21.z" version, which is one of the valid target versions: 4.21.0, 4.21.z
  • bug has dependents

Requesting review from QA contact:
/cc @melvinjoseph86

This pull request references Jira Issue OCPBUGS-87207, which is valid.

7 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target version (4.20.z) matches configured target version for branch (4.20.z)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)
  • release note text is set and does not match the template
  • dependent bug Jira Issue OCPBUGS-56281 is in the state Closed (Done), which is one of the valid states (VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA))
  • dependent Jira Issue OCPBUGS-56281 targets the "4.21.0" version, which is one of the valid target versions: 4.21.0, 4.21.z
  • bug has dependents

No GitHub users were found matching the public email listed for the QA contact in Jira (iamin@redhat.com), skipping review request.

Details

In response to this:

Backport Gateway API test improvements to enable running on vSphere and baremetal platforms.

Backported PRs

  1. NE-2090: adding one GIE automation test #30189 (Dependency)
  2. OCPBUGS-56281: gatewayapicontroller: Clean up resources when done #29900 (Dependency)
  3. OCPBUGS-56281: gatewayapicontroller: Use dynamic client for OLM #30397 (Dependency)
  4. NE-2056: auto negative tests of gatewayapicontroller #30599 (Dependency)
  5. NE-2292: Skip OLM tests when GatewayAPIWithoutOLM enabled #30896 (Dependency)
  6. NE-2292: Tests for gatewayAPIWithoutOLM featuregate graduation #30905 (Dependency)
  7. NE-2422: Skip Gateway API tests on dual-stack clusters #30942 (Dependency)
  8. NE-2520: Enable Gateway API tests on vSphere and baremetal #30946 (Target)

Why

The primary goal is to backport #30946 which enables Gateway API e2e tests to run on vSphere and baremetal platforms by gracefully handling missing LoadBalancer and DNS capabilities. PRs #30189 through #30942 are prerequisites that #30946 builds on — they restructure the test file, add constants and error handling, introduce GIE and negative tests, switch to dynamic client for OLM, and add feature gate handling that #30946 depends on. Including the full chain ensures clean cherry-picks and avoids future merge conflicts if any of these changes need to be backported independently later.

The GatewayAPIWithoutOLM feature gate tests are included but will not run — isNoOLMFeatureGateEnabled returns false early because the feature gate is not defined in openshift/api for this release branch. Without this early return, the feature gate lookup treats an undefined gate as a GA default and returns true, which would run the No OLM tests against a cluster using OLM-based provisioning and fail.

Dependency

All cherry-picks applied cleanly (only vendor/modules.txt needed ours-resolution for k8s fork hashes).

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@gcs278
Copy link
Copy Markdown
Contributor Author

gcs278 commented Jun 5, 2026

Just want to make sure we don't break tech preview tests:

/test e2e-gcp-ovn-techpreview

@gcs278
Copy link
Copy Markdown
Contributor Author

gcs278 commented Jun 5, 2026

I'm going volunteer @rikatz for this one since it's part of your on-prem support effort:
/assign @rikatz

Verify Gateway API CRDs and ensure required CRDs should already be installed'
- featureGate: GatewayAPIController
tests:
- testName: '[sig-network-edge][OCPFeatureGate:GatewayAPIController][Feature:Router][apigroup:gateway.networking.k8s.io]
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hum this test is new and related just with GatewayAPIController, but not with noOLM (some below are as well), was this intended?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(per the commits it seems so)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea this test-reporting file was abandoned in 4.22 it looks like, but is a requirement for 4.21 (verify fails).

It's intentional - these new tests add valuable coverage:

Will run (gated on GatewayAPIController)

  1. Ensure GIE is enabled after creating an inferencePool CRD (NE-2090: adding one GIE automation test #30189) — CIO GIE logic already on release-4.20 (35c0143b7), this adds missing test coverage.

  2. Ensure istiod deployment and the istio could be deleted and then get recreated [Serial] (NE-2520: Enable Gateway API tests on vSphere and baremetal #30946) — Tests existing CIO reconciliation logic.

  3. Ensure gateway LB service and dnsrecords could be deleted and then get recreated [Serial] (NE-2520: Enable Gateway API tests on vSphere and baremetal #30946) — Tests existing CIO reconciliation logic. Part of the target PR for this backport.

Won't run (gated on GatewayAPIWithoutOLM — not defined in 4.20)

4-8. Five noOLM tests from #30599, #30896, #30905 — dead code until the openshift/api backport lands. Included because they're part of the cherry-pick chain.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added to the PR description BTW

@gcs278
Copy link
Copy Markdown
Contributor Author

gcs278 commented Jun 5, 2026

Kinda of makes me nervous there isn't a tide merge query job 🤔 I think that means this will easily merge with an LGTM? Not even sure how that works. I'll hold until reviews are done just in case.

/hold

@openshift-ci openshift-ci Bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 5, 2026
@gcs278
Copy link
Copy Markdown
Contributor Author

gcs278 commented Jun 5, 2026

nice tide is back
/unhold

@openshift-ci openshift-ci Bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 5, 2026
@rikatz
Copy link
Copy Markdown
Member

rikatz commented Jun 5, 2026

Cherry-pick looks clean and brings some more e2e tests to Gateway API on 4.20

/lgtm
/approve
Thanks!

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Jun 5, 2026
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Jun 5, 2026

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: gcs278, rikatz
Once this PR has been reviewed and has the lgtm label, please assign stbenjam for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Jun 5, 2026

@gcs278: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-aws-ovn-serial-2of2 a76f868 link true /test e2e-aws-ovn-serial-2of2

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@melvinjoseph86
Copy link
Copy Markdown
Contributor

PR 2871 changes is verified as below:-
➜ oc get clusterversion
NAME VERSION AVAILABLE PROGRESSING SINCE STATUS
version 4.20.0-0-2026-06-06-162411-test-ci-ln-iskitkk-latest True False 4m45s Cluster version is 4.20.0-0-2026-06-06-162411-test-ci-ln-iskitkk-latest

➜ oc get featuregate cluster -o yaml
apiVersion: config.openshift.io/v1
kind: FeatureGate
metadata:
annotations:
include.release.openshift.io/self-managed-high-availability: "true"
creationTimestamp: "2026-06-06T16:42:23Z"
generation: 1
name: cluster
resourceVersion: "720"
uid: caedc2b4-27e1-4504-bd41-5b0679bf40b3
spec: {}
status:
featureGates:

  • disabled:
    • name: AWSClusterHostedDNS
    • name: AWSClusterHostedDNSInstall
    • name: AWSDedicatedHosts
    • name: AWSServiceLBNetworkSecurityGroup
    • name: AutomatedEtcdBackup
    • name: AzureClusterHostedDNSInstall
    • name: AzureDedicatedHosts
    • name: AzureMultiDisk
    • name: BootImageSkewEnforcement
    • name: BootcNodeManagement
    • name: ClusterAPIInstall
    • name: ClusterAPIInstallIBMCloud
    • name: ClusterMonitoringConfig
    • name: ClusterVersionOperatorConfiguration
    • name: DNSNameResolver
    • name: DualReplica
    • name: DyanmicServiceEndpointIBMCloud
    • name: DynamicResourceAllocation
    • name: EtcdBackendQuota
    • name: EventedPLEG
    • name: Example
    • name: Example2
    • name: ExternalSnapshotMetadata
    • name: GCPClusterHostedDNS
    • name: GCPClusterHostedDNSInstall
    • name: GCPCustomAPIEndpoints
    • name: GCPCustomAPIEndpointsInstall
    • name: GatewayAPIWithoutOLM
      <-----snip---->

@melvinjoseph86
Copy link
Copy Markdown
Contributor

confirmed that the OLM gatewayapicontroller tests pass on a vsphere clusterbot cluster.

    "name": "[sig-network-edge][OCPFeatureGate:GatewayAPIController][Feature:Router][apigroup:gateway.networking.k8s.io] Ensure OSSM and OLM related resources are created after creating GatewayClass [Suite:openshift/conformance/parallel]",
    "lifecycle": "blocking",
    "duration": 82846,
    "startTime": "2026-06-07 09:49:11.821148 UTC",
    "endTime": "2026-06-07 09:50:34.667792 UTC",
    "result": "passed",
    "name": "[sig-network-edge][OCPFeatureGate:GatewayAPIController][Feature:Router][apigroup:gateway.networking.k8s.io] Ensure default gatewayclass is accepted [Suite:openshift/conformance/parallel]",
    "lifecycle": "blocking",
    "duration": 25637,
    "startTime": "2026-06-07 09:53:24.090919 UTC",
    "endTime": "2026-06-07 09:53:49.728478 UTC",
    "result": "passed",
    "name": "[sig-network-edge][OCPFeatureGate:GatewayAPIController][Feature:Router][apigroup:gateway.networking.k8s.io] Ensure custom gatewayclass can be accepted [Suite:openshift/conformance/parallel]",
    "lifecycle": "blocking",
    "duration": 49927,
    "startTime": "2026-06-07 09:55:27.470736 UTC",
    "endTime": "2026-06-07 09:56:17.398503 UTC",
    "result": "passed",
    "name": "[sig-network-edge][OCPFeatureGate:GatewayAPIController][Feature:Router][apigroup:gateway.networking.k8s.io] Ensure LB, service, and dnsRecord are created for a Gateway object [Suite:openshift/conformance/parallel]",
    "lifecycle": "blocking",
    "duration": 36830,
    "startTime": "2026-06-07 09:57:30.412629 UTC",
    "endTime": "2026-06-07 09:58:07.243129 UTC",
    "result": "passed"
    "name": "[sig-network-edge][OCPFeatureGate:GatewayAPIController][Feature:Router][apigroup:gateway.networking.k8s.io] Ensure HTTPRoute object is created [Suite:openshift/conformance/parallel]",
    "lifecycle": "blocking",
    "duration": 50790,
    "startTime": "2026-06-07 09:58:41.228870 UTC",
    "endTime": "2026-06-07 09:59:32.019113 UTC",
    "result": "passed",
    "name": "[sig-network-edge][OCPFeatureGate:GatewayAPIController][Feature:Router][apigroup:gateway.networking.k8s.io] Ensure GIE is enabled after creating an inferencePool CRD [Suite:openshift/conformance/parallel]",
    "lifecycle": "blocking",
    "duration": 31871,
    "startTime": "2026-06-07 10:00:00.396897 UTC",
    "endTime": "2026-06-07 10:00:32.268756 UTC",
    "result": "passed",
    "name": "[sig-network-edge][OCPFeatureGate:GatewayAPIController][Feature:Router][apigroup:gateway.networking.k8s.io] Ensure istiod deployment and the istio could be deleted and then get recreated [Serial] [Suite:openshift/conformance/serial]",
    "lifecycle": "blocking",
    "duration": 29947,
    "startTime": "2026-06-07 10:12:19.253741 UTC",
    "endTime": "2026-06-07 10:12:49.201297 UTC",
    "result": "passed",

    [SKIPPED] Skipping LoadBalancer and DNS deletion test - platform does not support these features
    In [It] at: github.com/openshift/origin/test/extended/router/gatewayapicontroller.go:496 @ 06/07/26 15:39:14.034
  ------------------------------

  Ran 0 of 1 Specs in 15.135 seconds
  SUCCESS! -- 0 Passed | 0 Failed | 0 Pending | 1 Skipped
[
  {
    "name": "[sig-network-edge][OCPFeatureGate:GatewayAPIController][Feature:Router][apigroup:gateway.networking.k8s.io] Ensure gateway loadbalancer service and dnsrecords could be deleted and then get recreated [Serial] [Suite:openshift/conformance/serial]",
    "lifecycle": "blocking",
    "duration": 15135,
    "startTime": "2026-06-07 10:09:00.264210 UTC",
    "endTime": "2026-06-07 10:09:15.399575 UTC",
    "result": "skipped",

/verified by @mjoseph

@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Jun 7, 2026
@openshift-ci-robot
Copy link
Copy Markdown

@melvinjoseph86: This PR has been marked as verified by @mjoseph.

Details

In response to this:

confirmed that the OLM gatewayapicontroller tests pass on a vsphere clusterbot cluster.

   "name": "[sig-network-edge][OCPFeatureGate:GatewayAPIController][Feature:Router][apigroup:gateway.networking.k8s.io] Ensure OSSM and OLM related resources are created after creating GatewayClass [Suite:openshift/conformance/parallel]",
   "lifecycle": "blocking",
   "duration": 82846,
   "startTime": "2026-06-07 09:49:11.821148 UTC",
   "endTime": "2026-06-07 09:50:34.667792 UTC",
   "result": "passed",
   "name": "[sig-network-edge][OCPFeatureGate:GatewayAPIController][Feature:Router][apigroup:gateway.networking.k8s.io] Ensure default gatewayclass is accepted [Suite:openshift/conformance/parallel]",
   "lifecycle": "blocking",
   "duration": 25637,
   "startTime": "2026-06-07 09:53:24.090919 UTC",
   "endTime": "2026-06-07 09:53:49.728478 UTC",
   "result": "passed",
   "name": "[sig-network-edge][OCPFeatureGate:GatewayAPIController][Feature:Router][apigroup:gateway.networking.k8s.io] Ensure custom gatewayclass can be accepted [Suite:openshift/conformance/parallel]",
   "lifecycle": "blocking",
   "duration": 49927,
   "startTime": "2026-06-07 09:55:27.470736 UTC",
   "endTime": "2026-06-07 09:56:17.398503 UTC",
   "result": "passed",
   "name": "[sig-network-edge][OCPFeatureGate:GatewayAPIController][Feature:Router][apigroup:gateway.networking.k8s.io] Ensure LB, service, and dnsRecord are created for a Gateway object [Suite:openshift/conformance/parallel]",
   "lifecycle": "blocking",
   "duration": 36830,
   "startTime": "2026-06-07 09:57:30.412629 UTC",
   "endTime": "2026-06-07 09:58:07.243129 UTC",
   "result": "passed"
   "name": "[sig-network-edge][OCPFeatureGate:GatewayAPIController][Feature:Router][apigroup:gateway.networking.k8s.io] Ensure HTTPRoute object is created [Suite:openshift/conformance/parallel]",
   "lifecycle": "blocking",
   "duration": 50790,
   "startTime": "2026-06-07 09:58:41.228870 UTC",
   "endTime": "2026-06-07 09:59:32.019113 UTC",
   "result": "passed",
   "name": "[sig-network-edge][OCPFeatureGate:GatewayAPIController][Feature:Router][apigroup:gateway.networking.k8s.io] Ensure GIE is enabled after creating an inferencePool CRD [Suite:openshift/conformance/parallel]",
   "lifecycle": "blocking",
   "duration": 31871,
   "startTime": "2026-06-07 10:00:00.396897 UTC",
   "endTime": "2026-06-07 10:00:32.268756 UTC",
   "result": "passed",
   "name": "[sig-network-edge][OCPFeatureGate:GatewayAPIController][Feature:Router][apigroup:gateway.networking.k8s.io] Ensure istiod deployment and the istio could be deleted and then get recreated [Serial] [Suite:openshift/conformance/serial]",
   "lifecycle": "blocking",
   "duration": 29947,
   "startTime": "2026-06-07 10:12:19.253741 UTC",
   "endTime": "2026-06-07 10:12:49.201297 UTC",
   "result": "passed",

   [SKIPPED] Skipping LoadBalancer and DNS deletion test - platform does not support these features
   In [It] at: github.com/openshift/origin/test/extended/router/gatewayapicontroller.go:496 @ 06/07/26 15:39:14.034
 ------------------------------

 Ran 0 of 1 Specs in 15.135 seconds
 SUCCESS! -- 0 Passed | 0 Failed | 0 Pending | 1 Skipped
[
 {
   "name": "[sig-network-edge][OCPFeatureGate:GatewayAPIController][Feature:Router][apigroup:gateway.networking.k8s.io] Ensure gateway loadbalancer service and dnsrecords could be deleted and then get recreated [Serial] [Suite:openshift/conformance/serial]",
   "lifecycle": "blocking",
   "duration": 15135,
   "startTime": "2026-06-07 10:09:00.264210 UTC",
   "endTime": "2026-06-07 10:09:15.399575 UTC",
   "result": "skipped",

/verified by @mjoseph

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

jira/severity-critical Referenced Jira bug's severity is critical for the branch this PR is targeting. jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. vendor-update Touching vendor dir or related files verified Signifies that the PR passed pre-merge verification criteria

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants