Skip to content

Store ephemeral cluster credentials in a Secret instead of the status#5124

Merged
openshift-merge-bot[bot] merged 1 commit into
openshift:mainfrom
amisstea:KFLUXVNGD-899
Apr 23, 2026
Merged

Store ephemeral cluster credentials in a Secret instead of the status#5124
openshift-merge-bot[bot] merged 1 commit into
openshift:mainfrom
amisstea:KFLUXVNGD-899

Conversation

@amisstea
Copy link
Copy Markdown
Contributor

@amisstea amisstea commented Apr 22, 2026

The kubeconfig and kubeAdminPassword were previously written directly to the EphemeralCluster status, exposing sensitive credentials in a resource that may be logged or cached. Instead, create a credentials Secret in the same namespace as the EphemeralCluster and reference it via a new status.secretRef field. The Secret is owned by the EphemeralCluster for automatic cleanup.

Jira: https://redhat.atlassian.net/browse/KFLUXVNGD-899

Assisted-by: Claude claude-opus-4-6

Summary by CodeRabbit

  • New Features

    • EphemeralCluster now uses Secret references (secretRef) for storing credentials instead of direct fields in the specification and status. Credentials are consolidated into a single Secret resource in the same namespace containing kubeconfig and optional kubeAdminPassword keys.
  • Chores

    • Updated controller reconciliation and test cases to support the new credential storage mechanism.

@openshift-merge-bot
Copy link
Copy Markdown
Contributor

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: automatic mode

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 22, 2026

Warning

Rate limit exceeded

@amisstea has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 3 minutes and 55 seconds before requesting another review.

Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 3 minutes and 55 seconds.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 5daadf31-6477-4061-a940-9cfe34727597

📥 Commits

Reviewing files that changed from the base of the PR and between 74cf3ce and 72ca748.

📒 Files selected for processing (4)
  • pkg/api/ephemeralcluster/v1/ci.openshift.io_ephemeralclusters.yaml
  • pkg/api/ephemeralcluster/v1/types.go
  • pkg/controller/ephemeralcluster/reconciler.go
  • pkg/controller/ephemeralcluster/reconciler_test.go

Walkthrough

The pull request updates the EphemeralCluster API to consolidate credential management by replacing direct kubeconfig and kubeAdminPassword fields with a single secretRef field. The reconciler logic now creates and manages a separate credentials Secret object rather than embedding credentials directly in the cluster status.

Changes

Cohort / File(s) Summary
API Schema & Type Definitions
pkg/api/ephemeralcluster/v1/ci.openshift.io_ephemeralclusters.yaml, pkg/api/ephemeralcluster/v1/types.go
Replaced kubeconfig and kubeAdminPassword fields with a single secretRef field in EphemeralCluster spec and status, referencing a Secret containing these credentials.
Controller Logic
pkg/controller/ephemeralcluster/reconciler.go
Updated secret fetching functions to create a credentials Secret via new createCredentialsSecret helper instead of storing credentials directly in status. Added credentialsSecretName function and idempotent secret creation with owner references and error handling.
Tests
pkg/controller/ephemeralcluster/reconciler_test.go
Removed expectations for embedded credentials in status and added assertions for credentials Secret creation, including verification of secret name and data contents.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 11 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (11 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The pull request title directly and accurately describes the main change: moving credential storage from the status field to a Secret, which is the core modification across the CRD schema, types, reconciliation logic, and tests.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed Test file uses standard Go table-driven tests with static t.Run() names containing no dynamic information, ensuring stable and deterministic test names across runs.
Test Structure And Quality ✅ Passed Tests use focused subtests validating single behaviors with fake clients, no real cluster resources, no indefinite waits, and clear error messages adhering to repository patterns.
Microshift Test Compatibility ✅ Passed This pull request does not add any new Ginkgo e2e tests. The codebase contains only standard Go unit tests using the testing package. A comprehensive search confirmed zero imports of github.com/onsi/ginkgo in the repository. The PR only modifies existing table-driven unit tests in reconciler_test.go to add an additional expected field (wantSecret *corev1.Secret). Since no new Ginkgo e2e tests are being introduced, the MicroShift test compatibility check is not applicable to this PR.
Single Node Openshift (Sno) Test Compatibility ✅ Passed The pull request does not add any new Ginkgo e2e tests. Changes are limited to controller logic and traditional Go unit tests without e2e test suites.
Topology-Aware Scheduling Compatibility ✅ Passed Pull request refactors ephemeral cluster credentials storage from direct embedding to Kubernetes Secret references with no scheduling constraints or topology-dependent configuration.
Ote Binary Stdout Contract ✅ Passed The pull request does not introduce any OTE Binary Stdout Contract violations as the modified code is a Kubernetes controller using logrus for stderr logging without direct stdout writes.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed PR modifies controller code and unit tests, not Ginkgo e2e tests. Custom check for IPv4/connectivity assumptions in e2e tests is not applicable.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci openshift-ci Bot requested review from deepsm007 and droslean April 22, 2026 19:32
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
pkg/controller/ephemeralcluster/reconciler_test.go (1)

1352-1363: Consider also asserting the credentials Secret's OwnerReference.

The test fetches the Secret and diffs only Data. Given that the whole motivation of this PR is "the Secret is owned by the EphemeralCluster to enable automatic cleanup," it would be valuable to assert that the created Secret carries the expected controller ownerref pointing at the EC (especially if the [SetControllerReference/ownership-verification changes suggested on reconciler.go] land). This prevents silent regressions where ownership is dropped.

♻️ Example addition
 			if tc.wantSecret != nil {
 				gotSecret := corev1.Secret{}
 				if err := client.Get(context.TODO(), types.NamespacedName{
 					Name:      tc.wantSecret.Name,
 					Namespace: tc.wantSecret.Namespace,
 				}, &gotSecret); err != nil {
 					t.Fatalf("get credentials secret: %s", err)
 				}
 				if diff := cmp.Diff(tc.wantSecret.Data, gotSecret.Data); diff != "" {
 					t.Errorf("unexpected credentials secret data: %s", diff)
 				}
+				if !metav1.IsControlledBy(&gotSecret, tc.ec) {
+					t.Errorf("credentials secret %s/%s is not controlled by ephemeralcluster %s",
+						gotSecret.Namespace, gotSecret.Name, tc.ec.Name)
+				}
 			}

(Note: the test ECs would need to be instantiated with a non-empty UID for IsControlledBy to work against the fake client, since the fake client does not always populate UIDs on WithObjects.)

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pkg/controller/ephemeralcluster/reconciler_test.go` around lines 1352 - 1363,
Test currently compares only Secret.Data (tc.wantSecret vs gotSecret) and misses
asserting the Secret's OwnerReference; update the test in reconciler_test.go to
also verify the Secret is owned by the EphemeralCluster by checking
gotSecret.OwnerReferences (or using metav1.IsControlledBy against the
EphemeralCluster instance used in the test), ensuring the expected controller
ownerRef (controller=true, UID matching the EC UID) is present; if necessary,
ensure the fake EphemeralCluster object used in the test has a non-empty UID so
IsControlledBy/ownerRef comparisons succeed, and reference tc.wantSecret,
gotSecret, and the EphemeralCluster test object when making the assertion.
pkg/controller/ephemeralcluster/reconciler.go (1)

619-624: Use controllerutil.SetControllerReference instead of hand-crafted OwnerReference.

The current approach hard-codes the GVK (ci.openshift.io/v1, EphemeralCluster) and omits the Controller: true and BlockOwnerDeletion: true fields. Without Controller: true, another controller can claim ownership of this Secret without collision, and metav1.IsControlledBy will return false. SetControllerReference resolves the GVK from the scheme and sets both fields properly, following the pattern already used elsewhere in the codebase (e.g., multiarchbuildconfig.go).

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pkg/controller/ephemeralcluster/reconciler.go` around lines 619 - 624,
Replace the hand-crafted OwnerReferences block with a call to
controllerutil.SetControllerReference to let the scheme resolve the GVK and set
Controller and BlockOwnerDeletion properly; specifically, in the code that
currently sets OwnerReferences for the Secret, call
controllerutil.SetControllerReference(ec, secret, r.scheme) (or r.Scheme if
that’s the reconciler field name) and remove the manual APIVersion/Kind/Name/UID
assignment, ensuring the controller-runtime controllerutil package is imported.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@pkg/controller/ephemeralcluster/reconciler_test.go`:
- Around line 1352-1363: Test currently compares only Secret.Data (tc.wantSecret
vs gotSecret) and misses asserting the Secret's OwnerReference; update the test
in reconciler_test.go to also verify the Secret is owned by the EphemeralCluster
by checking gotSecret.OwnerReferences (or using metav1.IsControlledBy against
the EphemeralCluster instance used in the test), ensuring the expected
controller ownerRef (controller=true, UID matching the EC UID) is present; if
necessary, ensure the fake EphemeralCluster object used in the test has a
non-empty UID so IsControlledBy/ownerRef comparisons succeed, and reference
tc.wantSecret, gotSecret, and the EphemeralCluster test object when making the
assertion.

In `@pkg/controller/ephemeralcluster/reconciler.go`:
- Around line 619-624: Replace the hand-crafted OwnerReferences block with a
call to controllerutil.SetControllerReference to let the scheme resolve the GVK
and set Controller and BlockOwnerDeletion properly; specifically, in the code
that currently sets OwnerReferences for the Secret, call
controllerutil.SetControllerReference(ec, secret, r.scheme) (or r.Scheme if
that’s the reconciler field name) and remove the manual APIVersion/Kind/Name/UID
assignment, ensuring the controller-runtime controllerutil package is imported.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: d2b68d1d-5bb1-4bf1-a352-658f1f62af93

📥 Commits

Reviewing files that changed from the base of the PR and between 8d45148 and 74cf3ce.

📒 Files selected for processing (4)
  • pkg/api/ephemeralcluster/v1/ci.openshift.io_ephemeralclusters.yaml
  • pkg/api/ephemeralcluster/v1/types.go
  • pkg/controller/ephemeralcluster/reconciler.go
  • pkg/controller/ephemeralcluster/reconciler_test.go

Comment thread pkg/controller/ephemeralcluster/reconciler.go
The kubeconfig and kubeAdminPassword were previously written directly
to the EphemeralCluster status, exposing sensitive credentials in a
resource that may be logged or cached. Instead, create a credentials
Secret in the same namespace as the EphemeralCluster and reference it
via a new status.secretRef field. The Secret is owned by the
EphemeralCluster for automatic cleanup.

Assisted-by: Claude claude-opus-4-6
Signed-off-by: amisstea <amisstea@redhat.com>
@amisstea
Copy link
Copy Markdown
Contributor Author

/assign @danilo-gemoli

The corresponding change on the Konflux side is in konflux-ci/crossplane-control-plane#81.

@openshift-merge-bot
Copy link
Copy Markdown
Contributor

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e

@openshift-merge-bot
Copy link
Copy Markdown
Contributor

Tests from second stage were triggered manually. Pipeline can be controlled only manually, until HEAD changes. Use command to trigger second stage.

3 similar comments
@openshift-merge-bot
Copy link
Copy Markdown
Contributor

Tests from second stage were triggered manually. Pipeline can be controlled only manually, until HEAD changes. Use command to trigger second stage.

@openshift-merge-bot
Copy link
Copy Markdown
Contributor

Tests from second stage were triggered manually. Pipeline can be controlled only manually, until HEAD changes. Use command to trigger second stage.

@openshift-merge-bot
Copy link
Copy Markdown
Contributor

Tests from second stage were triggered manually. Pipeline can be controlled only manually, until HEAD changes. Use command to trigger second stage.


existing := corev1.Secret{}
err := r.masterClient.Get(ctx, types.NamespacedName{Name: secretName, Namespace: ec.Namespace}, &existing)
if err != nil && !kerrors.IsNotFound(err) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it's me, but I find this variation easier to reason about (even though it's a bit more nested):

if err != nil {
	if !kerrors.IsNotFound(err) {
		return fmt.Errorf("get credentials secret: %w", err)
	} else {
		if !v1.IsControlledBy(&existing, ec) {
			return fmt.Errorf("credentials secret %s/%s exists but is not owned by ephemeralcluster %s (uid=%s)",
				ec.Namespace, secretName, ec.Name, ec.UID)
		}
		return nil
	}
}

@danilo-gemoli
Copy link
Copy Markdown
Contributor

/lgtm

@danilo-gemoli
Copy link
Copy Markdown
Contributor

/override ci/prow/e2e

@openshift-ci openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Apr 23, 2026
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Apr 23, 2026

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: amisstea, danilo-gemoli

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 23, 2026
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Apr 23, 2026

@danilo-gemoli: Overrode contexts on behalf of danilo-gemoli: ci/prow/e2e

Details

In response to this:

/override ci/prow/e2e

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@danilo-gemoli
Copy link
Copy Markdown
Contributor

/override ci/prow/images

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Apr 23, 2026

@danilo-gemoli: Overrode contexts on behalf of danilo-gemoli: ci/prow/images

Details

In response to this:

/override ci/prow/images

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Apr 23, 2026

@amisstea: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/breaking-changes 72ca748 link false /test breaking-changes

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-merge-bot openshift-merge-bot Bot merged commit 0dc0fdf into openshift:main Apr 23, 2026
16 of 17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants