feat(e2e): make e2e:kubernetes work transparently on OpenShift - #3183
Open
jgarciao wants to merge 2 commits into
Open
feat(e2e): make e2e:kubernetes work transparently on OpenShift#3183jgarciao wants to merge 2 commits into
jgarciao wants to merge 2 commits into
Conversation
Running `mise run e2e:kubernetes` on OpenShift required manual namespace
creation, SCC grants, Helm value overrides, and cleanup. A separate
`e2e:openshift` task existed but only checked pod readiness without
running the Rust e2e test suite, and even with the suite wired up the
SSH-relay `sandbox connect` path stalled to the ready timeout because
`kubectl port-forward` cannot carry round-trip-heavy SSH over the
internet.
The harness now auto-detects OpenShift via the `route.openshift.io` API
group and, on OpenShift, both configures the cluster and switches the
gateway transport automatically:
- Drives the gateway through a passthrough OpenShift Route secured with
mandatory mTLS instead of port-forward, so the connect suites
(live_policy_update, port_forward, sync, connect-based
sandbox_lifecycle, settings_management) actually pass. Computes the
Route host from the cluster ingress domain, extracts client mTLS
material from the openshell-client-tls secret, waits for the Route to
serve mTLS, asserts a certless caller is rejected at the TLS
handshake, and registers an mTLS CLI gateway pointing at the Route.
- Applies an SCC-compatible Helm values overlay that removes hardcoded
runAsUser/fsGroup, letting OpenShift assign UIDs from the namespace
range.
- Grants the privileged SCC to openshell-sandbox before Helm install
and removes it during cleanup.
- Grants the anyuid SCC to the PostgreSQL fixture service account in
DB scenarios and removes it during cleanup.
- All oc commands use --context to target the correct cluster.
The OpenShift e2e overlay (ci/values-openshift-e2e.yaml) turns TLS back
on, enables the Route, promotes the cert-verified caller to a dev
principal, and forces `image.pullPolicy`/`supervisor.image.pullPolicy`
to Always so runs against the `latest` upstream image use it instead of
a stale copy cached on the cluster nodes. Every OpenShift branch is
gated on OPENSHIFT_DETECTED, so the vanilla-Kubernetes port-forward path
is unchanged.
The Helm template for podSecurityContext is wrapped with {{- with }} so
null values omit the block instead of rendering invalid YAML.
The separate e2e:openshift task and e2e-openshift.sh script are removed
since e2e:kubernetes now covers OpenShift.
TESTING.md is updated with Kubernetes e2e documentation including
OpenShift auto-detection, dropping the e2e-host-gateway feature on
remote clusters, pinning IMAGE_TAG when the CLI and image versions
differ, task variants, and environment variables.
The debug-openshell-cluster skill gains an OpenShift platform row and
two SCC failure patterns (gateway rejected over hardcoded runAsUser,
sandbox missing the privileged SCC) covering the SCC handling and
podSecurityContext behavior this change introduces.
Signed-off-by: Jorge Garcia Oncins <jgarciao@redhat.com>
Track the anyuid SCC grant for the PostgreSQL fixture with a dedicated OPENSHIFT_POSTGRES_SCC_GRANTED flag set before the fixture apply, so a failed apply no longer leaks the binding; cleanup now revokes it whenever the grant succeeded, independent of deploy state. Validate the Route server cert in the certless security gate (curl --cacert instead of -k) and classify curl's exit code so only a TLS client-auth rejection (35/56) counts as the expected certless rejection; an unrelated DNS/timeout/TLS failure now fails loudly instead of masking a potential mTLS hole. Raise the OpenShift Route timeout in the e2e overlay. The default HAProxy Route timeout is 30s, which severed long-lived transfers (large sandbox upload/download, SSH-relay `sandbox connect`) mid-stream and failed the sync e2e tests. Set both haproxy.router.openshift.io/timeout and timeout-tunnel to 300s: a passthrough Route proxies in TCP mode, so timeout-tunnel governs the established tunnel while timeout covers the pre-tunnel phase. Document the OpenShift transport exception, oc prerequisites and SCC grants, and make the skopeo tag-check example copy-safe in TESTING.md. Signed-off-by: Jorge Garcia Oncins <jgarciao@redhat.com>
jgarciao
requested review from
a team,
derekwaynecarr,
mrunalp and
sjenning
as code owners
September 4, 2026 10:30
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Makes
mise run e2e:kubernetesrun transparently on OpenShift. When the target cluster exposes theroute.openshift.ioAPI group, the harness auto-detects OpenShift and adapts the deployment and transport: it applies SCC-compatible Helm overrides, grants the required SCCs, and drives the gateway through a passthrough Route secured with mandatory mTLS instead ofkubectl port-forward(which stalls the SSH-relaysandbox connectpath). The previous standalone, weakere2e:openshifttask is removed. Vanilla-Kubernetes behavior is unchanged — every OpenShift branch is gated onOPENSHIFT_DETECTED.Related Issue
Closes #2956
Closes #3033
Changes
e2e/with-kube-gateway.shvia theroute.openshift.ioAPI group; all OpenShift-specific behavior is gated onOPENSHIFT_DETECTEDso the vanilla-Kubernetes path is untouched.kubectl port-forward, which stalls the round-trip-heavy SSH-relaysandbox connectsuites.privilegedSCC toopenshell-sandboxandanyuidto the PostgreSQL fixture; track grants with flags recorded before the operation socleanup()revokes them even on failure or interrupt. Extracted client mTLS material is removed on cleanup.ci/values-openshift-scc.yaml(drops hardcodedrunAsUser/fsGroupso restricted-v2 injects the namespace UID/GID range) andci/values-openshift-e2e.yaml(Route/mTLS transport,Alwayspull policy, raised HAProxy Route timeouts for long-lived transfers).securityContextin_gateway-workload.tplwith{{- with .Values.podSecurityContext }}so the SCC overlay can null it out.e2e:openshifttask ande2e/rust/e2e-openshift.sh(gateway-readiness-only), folding its coverage into the unified path.start_grpc_portforward/start_health_portforward/scenario_record_failurehelpers.TESTING.md; add OpenShift SCC/Route troubleshooting to thedebug-openshell-clusterskill.Testing
mise run pre-commitpassesmise run e2e:kubernetesagainst a live OpenShift cluster: SCC grants applied and revoked, passthrough Route serves mTLS, certless request rejected at the TLS handshake, and the SSH-relaysandbox connectsuites pass over the Route.OPENSHIFT_DETECTED).Checklist