Skip to content

Add aws-pod-identity-webhook e2e tls cases#8953

Open
gangwgr wants to merge 1 commit into
openshift:mainfrom
gangwgr:add-new-test
Open

Add aws-pod-identity-webhook e2e tls cases#8953
gangwgr wants to merge 1 commit into
openshift:mainfrom
gangwgr:add-new-test

Conversation

@gangwgr

@gangwgr gangwgr commented Jul 7, 2026

Copy link
Copy Markdown

Summary by CodeRabbit

  • Tests
    • Expanded end-to-end TLS coverage to verify multiple control-plane components, including additional webhook TLS checks and broader scenario handling.
    • Centralized minimum TLS version validation using cluster configuration and component startup settings, with updated expectations for TLS 1.2/1.3 acceptance and rejection, including modern/min TLS transitions.
    • Improved test consistency by running standardized in-pod TLS negotiation checks with conditional behavior for AWS vs non-AWS endpoints.
  • CI / Workflows
    • Added a new sequential control-plane-tls step to the Azure upgrade-and-chaos workflow, producing JUnit output.

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 7, 2026
@openshift-ci

openshift-ci Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@coderabbitai

coderabbitai Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The PR refactors the control-plane TLS E2E test to use shared component metadata and reusable helpers for pod lookup, in-pod TLS probing, protocol assertions, and minTLSVersion inspection from either a ConfigMap or deployment flag. The scenario coverage is updated to use these helpers across TLS profile transitions and downgrade checks, and the top-level describe scope now includes aws-pod-identity-webhook. The Azure upgrade-and-chaos lifecycle adds a new control-plane-tls sequential test group between upgrade and etcd-chaos.

Sequence Diagram(s)

sequenceDiagram
    participant TLSScenario
    participant KubernetesAPI
    participant ComponentPod
    TLSScenario->>KubernetesAPI: Find running component pod
    KubernetesAPI-->>TLSScenario: Return pod metadata
    TLSScenario->>ComponentPod: Execute openssl s_client
    ComponentPod-->>TLSScenario: Return TLS handshake output
    TLSScenario->>KubernetesAPI: Read ConfigMap or Deployment TLS minimum
    KubernetesAPI-->>TLSScenario: Return configured minimum TLS version
    TLSScenario->>TLSScenario: Validate acceptance or rejection expectations
Loading

Suggested reviewers: sjenning, cblecker


Important

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

❌ Failed checks (1 error)

Check name Status Explanation Resolution
No-Sensitive-Data-In-Logs ❌ Error expectTLSConnectionResult now embeds raw openssl s_client output in accept-path failures, which can leak cert hostnames/internal details. Remove raw result from failure text or redact cert/hostname fields; keep only a short protocol/rejection summary.
✅ Passed checks (10 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title matches a real part of the change by adding aws-pod-identity-webhook e2e TLS coverage, though it omits the broader refactor.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed All Ginkgo titles in the changed test file are static strings; no generated/dynamic values appear in test names, and azure.go adds no titles.
Test Structure And Quality ✅ Passed PASS — The new Ginkgo tests use scoped setup/cleanup, explicit Eventually timeouts, and added failure messages; the Azure matrix step is a simple addition.
Topology-Aware Scheduling Compatibility ✅ Passed PR only refactors e2e TLS tests and adds an Azure test-group entry; no manifests/controllers or scheduling constraints were introduced.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed New tests use cluster-internal pod execs and localhost, with no hardcoded IPv4 addresses or external/public connectivity.
No-Weak-Crypto ✅ Passed Changed files only add TLS 1.2/1.3 test helpers and an Azure test step; no MD5/SHA1/DES/RC4/3DES/Blowfish/ECB, custom crypto, or secret comparisons found.
Container-Privileges ✅ Passed The PR only changes Go test/lifecycle code; no K8s manifests or privilege fields (privileged, hostPID, hostNetwork, hostIPC, allowPrivilegeEscalation, SYS_ADMIN) were introduced.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@gangwgr
gangwgr marked this pull request as ready for review July 7, 2026 13:19
@openshift-ci openshift-ci Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 7, 2026
@gangwgr gangwgr changed the title Add aws-pod-identity-webhook e2e tls cases [WIP]Add aws-pod-identity-webhook e2e tls cases Jul 7, 2026
@openshift-ci openshift-ci Bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. area/testing Indicates the PR includes changes for e2e testing and removed do-not-merge/needs-area labels Jul 7, 2026
@openshift-ci

openshift-ci Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: gangwgr
Once this PR has been reviewed and has the lgtm label, please assign muraee for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@gangwgr

gangwgr commented Jul 7, 2026

Copy link
Copy Markdown
Author

/pipeline-required

@openshift-ci
openshift-ci Bot requested review from cblecker and sjenning July 7, 2026 13:21
@gangwgr

gangwgr commented Jul 7, 2026

Copy link
Copy Markdown
Author

/pipeline required

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-aks-4-22
/test e2e-aws-4-22
/test e2e-aks
/test e2e-aws
/test e2e-aws-upgrade-hypershift-operator
/test e2e-azure-v2-self-managed
/test e2e-kubevirt-aws-ovn-reduced
/test e2e-v2-aws
/test e2e-v2-gke

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/e2e/v2/tests/control_plane_tls_operator_test.go`:
- Around line 142-153: The ConfigMap check in the `configMapName` branch is
silently returning on `apierrors.IsNotFound`, which can make the `Eventually`
assertion pass vacuously. Update this path to fail loudly like the
`deploymentName` branch in `control_plane_tls_operator_test.go` by asserting on
the `mgmtClient.Get` result instead of returning early, and keep the
`cm.Data["config.yaml"]` minTLSVersion check tied to a successfully retrieved
ConfigMap so a missing object causes the test to fail.
- Around line 88-105: The getFirstRunningPod helper currently assumes
podList.Items[0] is the running pod, which can fail during rollouts because list
ordering is not guaranteed. Update getFirstRunningPod to iterate over all pods
returned by mgmtClient.List and return the first pod whose Status.Phase is
PodRunning, while keeping the existing error behavior when no pods are found or
none are running. Use the existing getFirstRunningPod function and
corev1.PodStatus checks to locate and fix the selection logic.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 3e67f4d0-3cb3-4ded-a4fe-82832c52fd9c

📥 Commits

Reviewing files that changed from the base of the PR and between dda6055 and c7efe37.

📒 Files selected for processing (1)
  • test/e2e/v2/tests/control_plane_tls_operator_test.go

Comment thread test/e2e/v2/tests/control_plane_tls_operator_test.go
Comment thread test/e2e/v2/tests/control_plane_tls_operator_test.go
@gangwgr

gangwgr commented Jul 7, 2026

Copy link
Copy Markdown
Author

/test all

@gangwgr

gangwgr commented Jul 7, 2026

Copy link
Copy Markdown
Author

/pipeline required

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-aks-4-22
/test e2e-aws-4-22
/test e2e-aks
/test e2e-aws
/test e2e-aws-upgrade-hypershift-operator
/test e2e-azure-v2-self-managed
/test e2e-kubevirt-aws-ovn-reduced
/test e2e-v2-aws
/test e2e-v2-gke

@hypershift-jira-solve-ci

Copy link
Copy Markdown
Contributor

AI Test Failure Analysis

Job: pull-ci-openshift-hypershift-main-e2e-aks | Build: 2074507846992531456 | Cost: $4.2398180000000005 | Failed step: hypershift-azure-run-e2e

View full analysis report


Generated by hypershift-analyze-e2e-failure post-step using Claude claude-opus-4-6

@cwbotbot

cwbotbot commented Jul 7, 2026

Copy link
Copy Markdown

Test Results

e2e-aws

Failed Tests

Total failed tests: 4

  • TestKarpenter
  • TestKarpenter/Main
  • TestKarpenter/Main/Parallel_provisioning_tests
  • TestKarpenter/Main/Parallel_provisioning_tests/OpenshiftEC2NodeClass_Kubelet_propagation

e2e-aks

@gangwgr gangwgr changed the title [WIP]Add aws-pod-identity-webhook e2e tls cases Add aws-pod-identity-webhook e2e tls cases Jul 8, 2026
@openshift-ci openshift-ci Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 8, 2026
@gangwgr

gangwgr commented Jul 8, 2026

Copy link
Copy Markdown
Author

/pipeline required

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-aks-4-22
/test e2e-aws-4-22
/test e2e-aks
/test e2e-aws
/test e2e-aws-upgrade-hypershift-operator
/test e2e-azure-v2-self-managed
/test e2e-kubevirt-aws-ovn-reduced
/test e2e-v2-aws
/test e2e-v2-gke

@codecov

codecov Bot commented Jul 8, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 44.11%. Comparing base (dda6055) to head (f66f075).
⚠️ Report is 85 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8953      +/-   ##
==========================================
+ Coverage   43.45%   44.11%   +0.65%     
==========================================
  Files         771      772       +1     
  Lines       95718    96227     +509     
==========================================
+ Hits        41597    42449     +852     
+ Misses      51234    50832     -402     
- Partials     2887     2946      +59     

see 52 files with indirect coverage changes

Flag Coverage Δ
cmd-support 38.24% <ø> (+1.11%) ⬆️
cpo-hostedcontrolplane 46.17% <ø> (+0.95%) ⬆️
cpo-other 45.13% <ø> (+0.03%) ⬆️
hypershift-operator 54.09% <ø> (+0.43%) ⬆️
other 32.12% <ø> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@gangwgr

gangwgr commented Jul 8, 2026

Copy link
Copy Markdown
Author

/retest

@hypershift-jira-solve-ci

Copy link
Copy Markdown
Contributor

AI Test Failure Analysis

Job: pull-ci-openshift-hypershift-main-e2e-aks | Build: 2074774735777959936 | Cost: $3.4668667500000003 | Failed step: hypershift-azure-run-e2e

View full analysis report


Generated by hypershift-analyze-e2e-failure post-step using Claude claude-opus-4-6

@gangwgr

gangwgr commented Jul 9, 2026

Copy link
Copy Markdown
Author

/retest

@openshift-ci openshift-ci Bot added the area/platform/azure PR/issue for Azure (AzurePlatform) platform label Jul 9, 2026
@gangwgr

gangwgr commented Jul 10, 2026

Copy link
Copy Markdown
Author

/pipeline required

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-aks-4-22
/test e2e-aws-4-22
/test e2e-aks
/test e2e-aws
/test e2e-aws-upgrade-hypershift-operator
/test e2e-azure-v2-self-managed
/test e2e-kubevirt-aws-ovn-reduced
/test e2e-v2-aws
/test e2e-v2-gke

@gangwgr

gangwgr commented Jul 10, 2026

Copy link
Copy Markdown
Author

/test e2e-v2-aws

@gangwgr

gangwgr commented Jul 13, 2026

Copy link
Copy Markdown
Author

/retest-required

@cblecker

Copy link
Copy Markdown
Member

Unresolved feedback from this review comment:

expectTLSConnectionResult has a vacuous-pass risk — when rejectConnection is false, ContainSubstring("") matches any string. A future tlsVersionExpectation{opensslFlag: "tls1_2"} (forgetting expectedProtocol) would silently pass. Please add a guard before the ContainSubstring assertion in the accept path:

g.Expect(expectation.expectedProtocol).NotTo(BeEmpty(),
    "expectedProtocol must be non-empty when rejectConnection is false")

@gangwgr

gangwgr commented Jul 15, 2026

Copy link
Copy Markdown
Author

/pipeline required

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-aks-4-22
/test e2e-aws-4-22
/test e2e-aks
/test e2e-aws
/test e2e-aws-upgrade-hypershift-operator
/test e2e-azure-v2-self-managed
/test e2e-kubevirt-aws-ovn-reduced
/test e2e-v2-aws
/test e2e-v2-gke

@gangwgr

gangwgr commented Jul 15, 2026

Copy link
Copy Markdown
Author

Unresolved feedback from this review comment:

expectTLSConnectionResult has a vacuous-pass risk — when rejectConnection is false, ContainSubstring("") matches any string. A future tlsVersionExpectation{opensslFlag: "tls1_2"} (forgetting expectedProtocol) would silently pass. Please add a guard before the ContainSubstring assertion in the accept path:

g.Expect(expectation.expectedProtocol).NotTo(BeEmpty(),
    "expectedProtocol must be non-empty when rejectConnection is false")

fixed

@gangwgr

gangwgr commented Jul 15, 2026

Copy link
Copy Markdown
Author

/pipeline required

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-aks-4-22
/test e2e-aws-4-22
/test e2e-aks
/test e2e-aws
/test e2e-aws-upgrade-hypershift-operator
/test e2e-azure-v2-self-managed
/test e2e-kubevirt-aws-ovn-reduced
/test e2e-v2-aws
/test e2e-v2-gke

@hypershift-jira-solve-ci

Copy link
Copy Markdown
Contributor

Now I have all the evidence I need. Let me produce the final report:

Test Failure Analysis Complete

Job Information

Test Failure Analysis

Error

[FAILED] container cluster-node-tuning-operator in pod cluster-node-tuning-operator-6f8f68769b-h4jhm has too many restarts (2 > 0)
Expected
    <int32>: 2
to be <=
    <int32>: 0

Summary

The e2e-v2-gke job failed due to a single test: [sig-hypershift][Jira:Hypershift][Feature:ControlPlaneWorkloads] Control Plane Workloads No crashing pods cluster-node-tuning-operator should have no crashing pods. The cluster-node-tuning-operator container in the hosted control plane namespace clusters-gcp-hc-35027f9d had restartCount: 2 at test time, exceeding the default toleration of 0. The container was not crash-looping with errors — it exited cleanly each time (exitCode: 0, reason: Completed), which is consistent with leader election loss or a graceful shutdown/restart cycle during initial cluster stabilization. The pod also experienced scheduling delays initially due to insufficient memory and untolerated taints on the GKE management cluster nodes. This failure is unrelated to the PR changes, which only modify TLS test files (control_plane_tls_operator_test.go, control_plane_pki_operator_test.go, and azure.go) and do not touch the control plane workloads test or the node-tuning-operator.

Root Cause

The cluster-node-tuning-operator pod restarted twice during the ~15-minute window between initial scheduling (08:09:17Z) and test execution (08:26:08Z). The container exited with exitCode: 0 (reason: Completed) each time — indicating graceful exits, not crashes. This is characteristic of leader election loss or operator initialization failures causing clean shutdown and restart during the hosted control plane stabilization period.

The pod initially had FailedScheduling events ("0/2 nodes are available: 1 Insufficient memory, 1 node(s) had untolerated taint(s)"), which delayed scheduling. Once scheduled, the container was created and started 3 times (first at 08:09:40Z, last at 08:24:20Z), with the penultimate instance running only ~67 seconds (08:22:58Z to 08:24:05Z) before exiting cleanly.

The test's isLeaderElectionFailure exemption mechanism (which fetches the last 10 lines of the previous container's logs and checks for "election lost") did not fire — most likely because the node-tuning-operator does not emit "election lost" in its log output when exiting, even though the exit behavior (exitCode 0, Completed) is functionally the same pattern. The cluster-node-tuning-operator is also missing from the crashTolerations map in the test, so it has a toleration of 0 restarts — making it vulnerable to any transient restart during control plane stabilization.

This is a flaky/infrastructure-related failure that is independent of the PR's changes.

Recommendations
  1. Add cluster-node-tuning-operator to the crashTolerations map in test/e2e/v2/tests/control_plane_workloads_test.go with a small toleration (e.g., 1 or 2) to account for expected restarts during hosted control plane stabilization, similar to the existing tolerations for gcp-cloud-controller-manager (1) and kubevirt-cloud-controller-manager (2).

  2. Retry / re-trigger the job — this failure is unrelated to the PR changes and is caused by transient cluster scheduling pressure on the GKE management cluster. A rerun is likely to pass.

  3. Consider enhancing the isLeaderElectionFailure check to also match other benign exit patterns, such as exitCode: 0 with reason: Completed, since the node-tuning-operator may not log "election lost" explicitly but still exits cleanly for the same reason.

Evidence
Evidence Detail
Failing test [sig-hypershift] Control Plane Workloads No crashing pods cluster-node-tuning-operator should have no crashing pods
Pod cluster-node-tuning-operator-6f8f68769b-h4jhm in namespace clusters-gcp-hc-35027f9d
restartCount 2 (toleration: 0)
Last exit exitCode: 0, reason: Completed (started 08:22:58Z, finished 08:24:05Z — ran 67s)
Current state at test time Running (started 08:24:20Z), ready: true
FailedScheduling events "0/2 nodes are available: 1 Insufficient memory, 1 node(s) had untolerated taint(s)"
BackOff event "Back-off restarting failed container" at 08:24:06Z
Container created/started 3 times (first: 08:09:40Z, last: 08:24:19Z/08:24:20Z)
ControlPlaneComponent status Available at 08:24:21Z (after 3rd container start)
CertHash annotation Not present on HostedControlPlane (cert rotation exemption did not apply)
PR files changed control_plane_tls_operator_test.go, control_plane_pki_operator_test.go, azure.go — none touch the failing test or node-tuning-operator
Test results 408 Passed, 1 Failed, 1 Pending, 561 Skipped out of 971 Specs

@gangwgr

gangwgr commented Jul 15, 2026

Copy link
Copy Markdown
Author

/pipeline required

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-aks-4-22
/test e2e-aws-4-22
/test e2e-aks
/test e2e-aws
/test e2e-aws-upgrade-hypershift-operator
/test e2e-azure-v2-self-managed
/test e2e-kubevirt-aws-ovn-reduced
/test e2e-v2-aws
/test e2e-v2-gke

@cblecker cblecker left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good refactoring to a reusable multi-component TLS test framework. The controlPlaneTLSComponent struct and shared helpers significantly reduce duplication. Previous review feedback (pod list ordering, vacuous-pass guards) was properly addressed.

A few items below — the IPv6 host:port and pod-restart verification gaps are the substantive ones.

@cblecker cblecker left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good refactoring to a reusable multi-component TLS test framework. The controlPlaneTLSComponent struct and shared helpers significantly reduce duplication. Previous review feedback (pod list ordering, vacuous-pass guards) was properly addressed.

A few items below — the IPv6 host:port and pod-restart verification gaps are the substantive ones.

return v2util.RunCommandInPod(ctx, mgmtKubeClient, mgmtRestConfig,
namespace, podName, containerName,
"sh", "-c",
fmt.Sprintf("timeout 5 openssl s_client -connect %s:%s -%s 2>&1 || true",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When connectViaPodIP is true, connectHost will be targetPod.Status.PodIP which can be an IPv6 address. The %s:%s formatting here would produce fd00::1:4443 instead of [fd00::1]:4443, breaking the openssl connect on IPv6/dual-stack clusters.

Use net.JoinHostPort(connectHost, port) instead:

fmt.Sprintf("timeout 5 openssl s_client -connect %s -%s 2>&1 || true",
    net.JoinHostPort(connectHost, port), tlsFlag)

})
})

It("aws-pod-identity-webhook should accept TLS 1.3 but reject TLS 1.2 with Modern profile", func() {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PKI operator tests (lines 506-545) wait for a new pod UID before probing TLS, but the webhook tests here go straight to verifyComponentTLSConnectivity without checking that the kube-apiserver pod has restarted with the new TLS config.

Since both Intermediate and Modern profiles accept TLS 1.3, the acceptance assertion can pass vacuously against a stale pod that hasn't picked up the new config yet. The same gap exists in the downgrade path at line 718.

Consider tracking the kube-apiserver pod UID before mutation and adding the same restart-wait pattern used for the PKI operator pod.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

// to the HostedCluster, the control-plane-pki-operator config reflects the correct minTLSVersion
// and that the control-plane-pki-operator's HTTPS endpoint enforces those TLS versions correctly.
func VerifyPKIOperatorTLSConfigTest(getTestCtx internal.TestContextGetter) {
When("control plane TLS configuration is modified", Ordered, Serial, func() {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: The old code had Label("Lifecycle") on this When block since it mutates the HostedCluster TLS profile. The label was dropped in the rename. While CI selection uses LabelFilter: "control-plane-pki-operator" from the Describe, the Lifecycle label serves as self-documentation that this block modifies cluster state.

Suggested change
When("control plane TLS configuration is modified", Ordered, Serial, func() {
When("control plane TLS configuration is modified", Ordered, Serial, Label("Lifecycle"), func() {

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This still doesn't have the label — line 372 adds a comment saying the omission is intentional for e2e-v2-aws compatibility, which contradicts the "updated" response here. The Lifecycle label should be present since this block mutates cluster state.

Extend control plane TLS operator e2e tests to verify
aws-pod-identity-webhook deployment flags and TLS connectivity
when the HostedCluster TLS profile changes.

Signed-off-by: gangwgr <rgangwar@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@gangwgr

gangwgr commented Jul 16, 2026

Copy link
Copy Markdown
Author

/pipeline required

@openshift-merge-bot

Copy link
Copy Markdown
Contributor

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-aks-4-22
/test e2e-aws-4-22
/test e2e-aks
/test e2e-aws
/test e2e-aws-upgrade-hypershift-operator
/test e2e-azure-v2-self-managed
/test e2e-kubevirt-aws-ovn-reduced
/test e2e-v2-aws
/test e2e-v2-gke

@gangwgr

gangwgr commented Jul 16, 2026

Copy link
Copy Markdown
Author

/test e2e-v2-gke

@openshift-ci

openshift-ci Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

@gangwgr: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Modern: &configv1.ModernTLSProfile{},
}
err = mgmtClient.Update(tc.Context, hostedCluster)
if apierrors.IsConflict(err) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The return on conflict inside Eventually(func(g Gomega){...}).Should(Succeed()) causes premature success — when the Update returns a conflict, no g.Expect assertion has failed for that iteration, so Eventually declares the callback successful and moves on without actually applying the update.

The same pattern appears at lines 635 and 758 (AfterAll cleanup). In the cleanup path, a conflict silently leaves the cluster with a mutated TLS profile.

Dropping the conflict guard and letting the error hit g.Expect(err).NotTo(HaveOccurred()) is the correct fix — the failed assertion triggers an Eventually retry, which re-gets the latest resourceVersion and reattempts the update.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/platform/azure PR/issue for Azure (AzurePlatform) platform area/testing Indicates the PR includes changes for e2e testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants