OCPQE-31708: Add all hosted cluster service NodePorts to squid allowed_ssl_ports by zhfeng · Pull Request #75701 · openshift/release

zhfeng · 2026-03-04T23:43:39Z

Summary

Fix squid proxy to allow all hosted cluster service NodePorts (oauth-openshift, konnectivity-server, ignition-server-proxy) in addition to the kube-apiserver port
The hypershift-agent-create-proxy step now reads SHARED_DIR/hosted_*_port files written by the hypershift-mce-agent-create-hostedcluster step and passes them to the remote squid configuration
Fixes [sig-auth][Feature:OAuthServer] well-known endpoint should be reachable conformance test failures across 4.16-4.21

Root Cause

The squid proxy's allowed_ssl_ports ACL only included the kube-apiserver NodePort (added by PR #75256). Other hosted cluster service NodePorts (e.g., oauth-openshift port 31846) were blocked, causing TCP_DENIED/403 when conformance tests tried to authenticate via the OAuth server through the proxy.

Evidence

From 4.16 build 2029114548413272064:

Squid access log: TCP_DENIED/403 CONNECT api...ostest.test.metalkube.org:31846 (OAuth port)
allowed_ssl_ports only had: 443 5000 6443 31778 (KAS port)
Squid DENIED timestamp exactly matches the test failure time

JIRA

https://issues.redhat.com/browse/OCPQE-31708

Test plan

Rehearse against a hypershift agent metal conformance job
Verify squid config includes all hosted cluster service NodePorts in allowed_ssl_ports
Verify [sig-auth][Feature:OAuthServer] well-known endpoint should be reachable test passes

openshift-ci-robot · 2026-03-04T23:43:43Z

@zhfeng: This pull request references OCPQE-31708 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the epic to target the "4.22.0" version, but no target version was set.

Details

In response to this:

Summary

Fix squid proxy to allow all hosted cluster service NodePorts (oauth-openshift, konnectivity-server, ignition-server-proxy) in addition to the kube-apiserver port

The hypershift-agent-create-proxy step now reads SHARED_DIR/hosted_*_port files written by the hypershift-mce-agent-create-hostedcluster step and passes them to the remote squid configuration

Fixes [sig-auth][Feature:OAuthServer] well-known endpoint should be reachable conformance test failures across 4.16-4.21

Root Cause

The squid proxy's allowed_ssl_ports ACL only included the kube-apiserver NodePort (added by PR #75256). Other hosted cluster service NodePorts (e.g., oauth-openshift port 31846) were blocked, causing TCP_DENIED/403 when conformance tests tried to authenticate via the OAuth server through the proxy.

Evidence

From 4.16 build 2029114548413272064:

Squid access log: TCP_DENIED/403 CONNECT api...ostest.test.metalkube.org:31846 (OAuth port)

allowed_ssl_ports only had: 443 5000 6443 31778 (KAS port)

Squid DENIED timestamp exactly matches the test failure time

JIRA

https://issues.redhat.com/browse/OCPQE-31708

Test plan

Rehearse against a hypershift agent metal conformance job

Verify squid config includes all hosted cluster service NodePorts in allowed_ssl_ports

Verify [sig-auth][Feature:OAuthServer] well-known endpoint should be reachable test passes

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

…d_ssl_ports

openshift-ci-robot · 2026-03-04T23:46:25Z

[REHEARSALNOTIFIER]
@zhfeng: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name	Repo	Type	Reason
pull-ci-openshift-origin-release-4.16-e2e-agent-disconnected-ovn-dualstack-metal3	openshift/origin	presubmit	Registry content changed
pull-ci-openshift-origin-release-4.16-e2e-agent-connected-ovn-ipv4-metal3	openshift/origin	presubmit	Registry content changed
pull-ci-openshift-origin-release-4.16-e2e-agent-connected-ovn-dualstack-metal3	openshift/origin	presubmit	Registry content changed
periodic-ci-openshift-hypershift-release-4.20-periodics-mce-e2e-agent-connected-ovn-dualstack-metal-conformance	N/A	periodic	Registry content changed
periodic-ci-openshift-openshift-tests-private-release-4.16-amd64-nightly-baremetalds-hypershift-agent-mce-guest-f28-destructive	N/A	periodic	Registry content changed
periodic-ci-openshift-hypershift-release-4.17-periodics-mce-e2e-agent-connected-ovn-ipv4-metal-conformance	N/A	periodic	Registry content changed
periodic-ci-openshift-hypershift-release-4.16-periodics-mce-e2e-agent-connected-ovn-dualstack-metal-conformance	N/A	periodic	Registry content changed
periodic-ci-openshift-openshift-tests-private-release-4.16-amd64-nightly-4.16-upgrade-from-stable-4.15-baremetalds-agent-hypershift-mce-compact-full-inplace-f14	N/A	periodic	Registry content changed
periodic-ci-openshift-hypershift-release-4.19-periodics-mce-e2e-agent-critical	N/A	periodic	Registry content changed
periodic-ci-openshift-openshift-tests-private-release-4.17-amd64-nightly-4.17-upgrade-from-stable-4.16-baremetalds-agent-hypershift-mce-compact-full-inplace-f14	N/A	periodic	Registry content changed
periodic-ci-openshift-hypershift-release-4.17-periodics-mce-e2e-agent-critical	N/A	periodic	Registry content changed
periodic-ci-openshift-openshift-tests-private-release-4.17-amd64-nightly-4.17-upgrade-from-stable-4.16-baremetalds-agent-hypershift-mce-inplace-f14	N/A	periodic	Registry content changed
periodic-ci-openshift-hypershift-release-4.21-periodics-mce-e2e-agent-connected-ovn-ipv4-metal-oadp	N/A	periodic	Registry content changed
periodic-ci-openshift-hypershift-release-4.19-periodics-mce-e2e-agent-connected-ovn-ipv4-manual-conformance	N/A	periodic	Registry content changed
periodic-ci-openshift-hypershift-release-4.17-periodics-mce-e2e-agent-connected-ovn-ipv4-metal-compact-conformance	N/A	periodic	Registry content changed
periodic-ci-openshift-hypershift-release-4.21-periodics-mce-e2e-agent-critical	N/A	periodic	Registry content changed
periodic-ci-openshift-hypershift-release-4.17-periodics-mce-e2e-agent-connected-ovn-ipv4-metal-oadp	N/A	periodic	Registry content changed
periodic-ci-openshift-hypershift-release-4.16-periodics-mce-e2e-agent-connected-ovn-ipv4-metal-oadp	N/A	periodic	Registry content changed
periodic-ci-openshift-openshift-tests-private-release-4.19-amd64-nightly-baremetalds-hypershift-agent-mce-n1minor-mgmt-f28	N/A	periodic	Registry content changed
periodic-ci-openshift-openshift-tests-private-release-4.18-amd64-nightly-4.18-eus-upgrade-from-4.16-baremetalds-agent-hypershift-mce-compact-full-inplace-f28	N/A	periodic	Registry content changed
periodic-ci-openshift-hypershift-release-4.21-periodics-mce-e2e-agent-connected-ovn-dualstack-metal-conformance	N/A	periodic	Registry content changed
periodic-ci-openshift-openshift-tests-private-release-4.18-amd64-nightly-4.18-upgrade-from-stable-4.17-baremetalds-agent-hypershift-mce-compact-full-inplace-f14	N/A	periodic	Registry content changed
periodic-ci-openshift-openshift-tests-private-release-4.19-amd64-nightly-baremetalds-hypershift-agent-mce-n1minor-guest-f28	N/A	periodic	Registry content changed
periodic-ci-openshift-openshift-tests-private-release-4.17-amd64-nightly-baremetalds-hypershift-agent-mce-disconnected-guest-f14-des	N/A	periodic	Registry content changed
periodic-ci-openshift-openshift-tests-private-release-4.16-amd64-nightly-baremetalds-hypershift-agent-mce-dualstack-mgmt-f14	N/A	periodic	Registry content changed

A total of 119 jobs have been affected by this change. The above listing is non-exhaustive and limited to 25 jobs.

A full list of affected jobs can be found here
Prior to this PR being merged, you will need to either run and acknowledge or opt to skip these rehearsals.

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

zhfeng · 2026-03-04T23:46:50Z

/pj-rehearse periodic-ci-openshift-hypershift-release-4.16-periodics-mce-e2e-agent-connected-ovn-ipv4-metal-conformance

openshift-ci-robot · 2026-03-04T23:46:52Z

@zhfeng: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

openshift-ci-robot · 2026-03-04T23:47:45Z

[REHEARSALNOTIFIER]
@zhfeng: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name	Repo	Type	Reason
pull-ci-openshift-origin-release-4.16-e2e-agent-disconnected-ovn-dualstack-metal3	openshift/origin	presubmit	Registry content changed
pull-ci-openshift-origin-release-4.16-e2e-agent-connected-ovn-ipv4-metal3	openshift/origin	presubmit	Registry content changed
pull-ci-openshift-origin-release-4.16-e2e-agent-connected-ovn-dualstack-metal3	openshift/origin	presubmit	Registry content changed
periodic-ci-openshift-hypershift-release-4.21-periodics-mce-e2e-agent-connected-ovn-ipv4-metal-oadp	N/A	periodic	Registry content changed
periodic-ci-openshift-hypershift-release-4.21-periodics-mce-e2e-agent-disconnected-ovn-ipv6-metal-conformance	N/A	periodic	Registry content changed
periodic-ci-openshift-hypershift-release-4.18-periodics-mce-e2e-agent-connected-ovn-ipv4-metal-conformance	N/A	periodic	Registry content changed
periodic-ci-openshift-openshift-tests-private-release-4.17-amd64-nightly-baremetalds-hypershift-agent-mce-dualstack-guest-f14-des	N/A	periodic	Registry content changed
periodic-ci-openshift-hypershift-release-4.20-periodics-mce-e2e-agent-critical	N/A	periodic	Registry content changed
periodic-ci-openshift-openshift-tests-private-release-4.16-amd64-nightly-4.16-upgrade-from-stable-4.15-baremetalds-agent-hypershift-mce-compact-full-inplace-f14	N/A	periodic	Registry content changed
periodic-ci-openshift-hypershift-release-4.20-periodics-mce-e2e-agent-connected-ovn-ipv4-manual-conformance	N/A	periodic	Registry content changed
periodic-ci-openshift-openshift-tests-private-release-4.19-amd64-nightly-baremetalds-hypershift-agent-mce-n1minor-mgmt-f28	N/A	periodic	Registry content changed
periodic-ci-openshift-hypershift-release-4.21-periodics-mce-e2e-agent-connected-ovn-ipv4-metal-conformance	N/A	periodic	Registry content changed
periodic-ci-openshift-hypershift-release-4.17-periodics-mce-e2e-agent-disconnected-ovn-ipv6-metal-conformance	N/A	periodic	Registry content changed
periodic-ci-openshift-openshift-tests-private-release-4.18-amd64-nightly-baremetalds-hypershift-agent-mce-disconnected-guest-f14	N/A	periodic	Registry content changed
periodic-ci-openshift-openshift-tests-private-release-4.16-amd64-nightly-4.16-upgrade-from-stable-4.16-baremetalds-agent-hypershift-mce-inplace-f60	N/A	periodic	Registry content changed
periodic-ci-openshift-hypershift-release-4.17-periodics-mce-e2e-agent-connected-ovn-ipv4-manual-conformance	N/A	periodic	Registry content changed
periodic-ci-openshift-hypershift-release-4.20-periodics-mce-e2e-agent-connected-ovn-ipv4-metal-conformance	N/A	periodic	Registry content changed
periodic-ci-openshift-openshift-tests-private-release-4.16-amd64-nightly-baremetalds-hypershift-agent-mce-dualstack-guest-f14-des	N/A	periodic	Registry content changed
periodic-ci-openshift-hypershift-release-4.21-periodics-mce-e2e-agent-connected-ovn-ipv4-manual-conformance	N/A	periodic	Registry content changed
periodic-ci-openshift-hypershift-release-4.21-periodics-mce-e2e-agent-connected-ovn-dualstack-metal-conformance	N/A	periodic	Registry content changed
periodic-ci-openshift-hypershift-release-4.18-periodics-mce-e2e-agent-connected-ovn-ipv4-metal-compact-conformance	N/A	periodic	Registry content changed
periodic-ci-openshift-openshift-tests-private-release-4.16-amd64-nightly-baremetalds-hypershift-agent-mce-disconnected-guest-f14-des	N/A	periodic	Registry content changed
periodic-ci-openshift-openshift-tests-private-release-4.20-amd64-nightly-4.20-upgrade-from-stable-4.20-baremetalds-agent-hypershift-mce-inplace-f60	N/A	periodic	Registry content changed
periodic-ci-openshift-hypershift-release-4.17-periodics-mce-e2e-agent-connected-ovn-dualstack-metal-conformance	N/A	periodic	Registry content changed
periodic-ci-openshift-openshift-tests-private-release-4.18-amd64-nightly-baremetalds-hypershift-agent-mce-guest-f14	N/A	periodic	Registry content changed

A total of 119 jobs have been affected by this change. The above listing is non-exhaustive and limited to 25 jobs.

A full list of affected jobs can be found here

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

zhfeng · 2026-03-05T06:43:47Z

pj-rehearse ack

zhfeng · 2026-03-05T06:47:06Z

/pj-rehearse ack

openshift-ci-robot · 2026-03-05T06:47:09Z

@zhfeng: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel.

heliubj18 · 2026-03-05T08:07:51Z

/lgtm

openshift-ci · 2026-03-05T08:09:16Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: heliubj18, zhfeng

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~ci-operator/step-registry/hypershift/agent/OWNERS~~ [heliubj18]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

…d_ssl_ports (openshift#75701)

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Mar 4, 2026

openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 4, 2026

OCPQE-31708: Add all hosted cluster service NodePorts to squid allowe…

20c4f9a

…d_ssl_ports

zhfeng force-pushed the fix-squid-proxy-oauth-port branch from fa07f3f to 20c4f9a Compare March 4, 2026 23:44

openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 4, 2026

openshift-ci bot requested review from eranco74 and sjenning March 4, 2026 23:45

openshift-ci-robot added the rehearsals-ack Signifies that rehearsal jobs have been acknowledged label Mar 5, 2026

openshift-ci bot assigned heliubj18 Mar 5, 2026

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Mar 5, 2026

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 5, 2026

openshift-merge-bot bot merged commit 903cee5 into openshift:main Mar 5, 2026
10 checks passed

coleenquadros pushed a commit to coleenquadros/release that referenced this pull request Mar 5, 2026

OCPQE-31708: Add all hosted cluster service NodePorts to squid allowe…

0210eb3

…d_ssl_ports (openshift#75701)

sdodson pushed a commit to sdodson/release that referenced this pull request Mar 8, 2026

OCPQE-31708: Add all hosted cluster service NodePorts to squid allowe…

c24ff0e

…d_ssl_ports (openshift#75701)

SeanZhao-redhat pushed a commit to SeanZhao-redhat/openshift-release that referenced this pull request Mar 9, 2026

OCPQE-31708: Add all hosted cluster service NodePorts to squid allowe…

c8de268

…d_ssl_ports (openshift#75701)

kasturinarra pushed a commit to kasturinarra/release that referenced this pull request Mar 11, 2026

OCPQE-31708: Add all hosted cluster service NodePorts to squid allowe…

cce76f3

…d_ssl_ports (openshift#75701)

amp-rh pushed a commit to amp-rh/openshift-release that referenced this pull request Mar 12, 2026

OCPQE-31708: Add all hosted cluster service NodePorts to squid allowe…

96a21d5

…d_ssl_ports (openshift#75701)

tareqalayan pushed a commit to tareqalayan/release that referenced this pull request Mar 13, 2026

OCPQE-31708: Add all hosted cluster service NodePorts to squid allowe…

80d41ae

…d_ssl_ports (openshift#75701)

qiliRedHat pushed a commit to qiliRedHat/release that referenced this pull request Mar 13, 2026

OCPQE-31708: Add all hosted cluster service NodePorts to squid allowe…

4634eaf

…d_ssl_ports (openshift#75701)

MayXuQQ pushed a commit to MayXuQQ/release that referenced this pull request Mar 17, 2026

OCPQE-31708: Add all hosted cluster service NodePorts to squid allowe…

ae33d61

…d_ssl_ports (openshift#75701)

kasturinarra pushed a commit to kasturinarra/release that referenced this pull request Mar 17, 2026

OCPQE-31708: Add all hosted cluster service NodePorts to squid allowe…

a9f50c2

…d_ssl_ports (openshift#75701)

sairameshv pushed a commit to sairameshv/release that referenced this pull request Mar 23, 2026

OCPQE-31708: Add all hosted cluster service NodePorts to squid allowe…

fe8848d

…d_ssl_ports (openshift#75701)

zhouying7780 pushed a commit to zhouying7780/release that referenced this pull request Mar 25, 2026

OCPQE-31708: Add all hosted cluster service NodePorts to squid allowe…

70a3d2c

…d_ssl_ports (openshift#75701)

rrasouli pushed a commit to rrasouli/release that referenced this pull request Mar 25, 2026

OCPQE-31708: Add all hosted cluster service NodePorts to squid allowe…

85cf039

…d_ssl_ports (openshift#75701)

anpingli pushed a commit to anpingli/release that referenced this pull request Mar 30, 2026

OCPQE-31708: Add all hosted cluster service NodePorts to squid allowe…

6f9a6b5

…d_ssl_ports (openshift#75701)

Conversation

zhfeng commented Mar 4, 2026

Summary

Root Cause

Evidence

JIRA

Test plan

Uh oh!

openshift-ci-robot commented Mar 4, 2026 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Root Cause

Evidence

JIRA

Test plan

Uh oh!

openshift-ci-robot commented Mar 4, 2026

Uh oh!

zhfeng commented Mar 4, 2026

Uh oh!

openshift-ci-robot commented Mar 4, 2026

Uh oh!

openshift-ci-robot commented Mar 4, 2026

Uh oh!

zhfeng commented Mar 5, 2026

Uh oh!

zhfeng commented Mar 5, 2026

Uh oh!

openshift-ci-robot commented Mar 5, 2026

Uh oh!

heliubj18 commented Mar 5, 2026

Uh oh!

openshift-ci bot commented Mar 5, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

openshift-ci-robot commented Mar 4, 2026 •

edited by openshift-ci bot

Loading