Skip to content

CNTRLPLANE-3237: Introduce kms to kms migration#2192

Open
ardaguclu wants to merge 1 commit intoopenshift:masterfrom
ardaguclu:kms-to-kms
Open

CNTRLPLANE-3237: Introduce kms to kms migration#2192
ardaguclu wants to merge 1 commit intoopenshift:masterfrom
ardaguclu:kms-to-kms

Conversation

@ardaguclu
Copy link
Copy Markdown
Member

@ardaguclu ardaguclu commented Apr 30, 2026

This PR introduces KMS to KMS migration described in https://github.com/openshift/enhancements/blob/master/enhancements/kube-apiserver/kms-encryption-foundations.md. KMS to KMS migration is triggered only when predefined set of fields are updated.

This PR paves the way for pre-flight checker. So that preflight-checker can be integrated to key controller.

Summary by CodeRabbit

  • New Features

    • Per-key KMS provider configurations are tracked and stored alongside encryption keys, improving KMS→KMS migration accuracy and cross-secret consistency.
  • Bug Fixes / Behavior

    • Controller now detects meaningful KMS provider changes (e.g., address, namespace, transit settings) and triggers key rollover; innocuous config changes no longer cause rotations.
  • Tests

    • Expanded unit and e2e coverage for provider-change detection, migration flows, no-op scenarios, and per-key consistency checks.

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Apr 30, 2026
@openshift-ci-robot
Copy link
Copy Markdown

openshift-ci-robot commented Apr 30, 2026

@ardaguclu: This pull request references CNTRLPLANE-3237 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "5.0.0" version, but no target version was set.

Details

In response to this:

This PR introduces KMS to KMS migration described in https://github.com/openshift/enhancements/blob/master/enhancements/kube-apiserver/kms-encryption-foundations.md. KMS to KMS migration is triggered only when predefined set of fields are updated.

This PR paves the way for pre-flight checker. So that preflight-checker can be integrated to key controller.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 30, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

Key controller and KMS helpers detect KMS provider-config changes that require key migration, create new encryption keys with internal reason "kms-provider-changed", and propagate per-key KMS provider configs into encryption config secrets; tests and e2e flows updated to cover migration vs non-migration deltas.

Changes

KMS provider + key lifecycle

Layer / File(s) Summary
Data Shape
pkg/operator/encryption/encryptiondata/config.go, pkg/operator/encryption/encryptiondata/secret.go, pkg/operator/encryption/state/types.go
Add Config.KMSProviders map[string]*configv1.KMSConfig; FromEncryptionState and FromSecret collect KMS provider configs; secret marshalling/unmarshalling stores provider entries under kms.ProviderConfigDataKey(keyID); state.KMSConfig.Provider type switched to *configv1.KMSConfig.
KMS Utilities
pkg/operator/encryption/kms/helpers.go, pkg/operator/encryption/kms/helpers_test.go
Add ProviderConfigDataKey / ProviderConfigKeyID helpers and NeedsNewKey(latest, current *configv1.KMSConfig) bool that returns true for Type changes or Vault KEK-relevant changes (VaultAddress, VaultNamespace, TransitMount, TransitKey); unit tests added for extraction and decision logic.
Core Controller Logic
pkg/operator/encryption/controllers/key_controller.go
needsNewKey now accepts currentApiServerEncryption configv1.APIServerEncryption; KMS branch calls kms.NeedsNewKey(...) and requests new key creation with internal reason "kms-provider-changed" when provider config differs; getCurrentModeReasonAndEncryptionConfig and generateKeySecret use configv1.APIServerEncryption.
Testing helpers & fixtures
pkg/operator/encryption/testing/helpers.go, pkg/operator/encryption/secrets/secrets_test.go
Introduce DefaultKMSProviderConfig and CreateEncryptionKeySecretWithCustomKMSConfig; tests use shared default/custom KMS provider configs instead of empty placeholders.
Unit & Integration Tests
pkg/operator/encryption/controllers/key_controller_test.go, pkg/operator/encryption/controllers/state_controller_test.go, pkg/operator/encryption/kms/helpers_test.go, pkg/operator/encryption/secrets/secrets_test.go
Controller tests add scenarios that recreate keys when Vault.VaultAddress or Vault.TransitKey change (assert internal-reason=secrets-kms-provider-changed and new key IDs), and no-op scenarios for non-migration fields (KMSPluginImage, AppRole secret name, TLS CABundle). State tests updated expected KMSProviders and TypeMeta.
E2E flow
test/e2e-encryption/encryption_test.go
Add verifyKMSProviders helper to assert per-key provider-config parity between encryption-config and encryption-key-...-<keyID> secrets; expand KMS→KMS migration step (VaultAddress change) and add non-migration plugin-image change; adjust expected key numbering and downstream expectations.

Sequence Diagram

sequenceDiagram
    autonumber
    participant Controller as Controller
    participant KMSHelpers as KMS Helpers
    participant SecretStore as Secret Store
    participant API as API Server
    Controller->>API: Read current APIServerEncryption (KMS config)
    Controller->>SecretStore: Read latest encryption keys
    Controller->>KMSHelpers: Compare latest key provider vs current KMS config
    KMSHelpers-->>Controller: NeedsNewKey? (true/false)
    alt NeedsNewKey == true
        Controller->>SecretStore: Create new encryption-key secret (internal-reason: kms-provider-changed)
        SecretStore-->>Controller: Confirm creation
        Controller->>API: Update encryption-config referencing new key
    else NeedsNewKey == false
        Controller-->>Controller: No key rotation
    end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 10 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Test Structure And Quality ⚠️ Warning E2E test violates single responsibility by combining multiple unrelated scenarios (KMS enable/disable/rotation/migration) in one 864-line test. Uses non-deterministic time.Sleep() waits. Split TestEncryptionIntegration into focused scenario tests. Replace time.Sleep() with wait.Poll mechanisms. Ensure all assertions include meaningful failure messages.
✅ Passed checks (10 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: introducing KMS-to-KMS migration functionality, which is the primary focus of all modifications across the codebase.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed All test names are stable and deterministic. Tests use static descriptive strings with no dynamic values, function calls, or generated identifiers.
Microshift Test Compatibility ✅ Passed No new Ginkgo e2e tests added. All tests use standard Go testing.T framework, not Ginkgo. MicroShift check is not applicable.
Single Node Openshift (Sno) Test Compatibility ✅ Passed This PR does not add any Ginkgo e2e tests. All test files use standard Go testing (testing.T). The custom check applies only to new Ginkgo tests and is therefore not applicable.
Topology-Aware Scheduling Compatibility ✅ Passed This PR modifies encryption controller and KMS logic without introducing scheduling constraints, deployment manifests, or topology-incompatible pod configurations.
Ote Binary Stdout Contract ✅ Passed Library package PR. OTE Stdout Contract applies only to test binaries. No process-level functions added.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed The PR does not introduce any Ginkgo e2e tests. All test files use standard Go testing.T. No IPv4 assumptions, external connectivity, or disconnected environment incompatibilities found.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@openshift-ci openshift-ci Bot requested review from dgrisonnet and p0lyn0mial April 30, 2026 07:21
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented Apr 30, 2026

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: ardaguclu
Once this PR has been reviewed and has the lgtm label, please assign p0lyn0mial for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ardaguclu
Copy link
Copy Markdown
Member Author

ardaguclu commented Apr 30, 2026

Due to the test function updates, #2163 is prerequisite for this PR (there is no dependency in terms of core functionality)
/hold

@openshift-ci openshift-ci Bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 30, 2026
@ardaguclu
Copy link
Copy Markdown
Member Author

/uncc @dgrisonnet
/cc @p0lyn0mial @bertinatto

@openshift-ci openshift-ci Bot requested review from bertinatto and removed request for dgrisonnet April 30, 2026 07:21
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (2)
pkg/operator/encryption/controllers/key_controller_test.go (1)

622-774: ⚡ Quick win

Consider one regression case for “existing KMS secret missing provider config”.

Given the new KMS comparison path, a test with an older/malformed KMS key secret lacking provider config would protect against panic regressions and assert fallback behavior (recreate key with kms-provider-changed).

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pkg/operator/encryption/controllers/key_controller_test.go` around lines 622
- 774, Add a regression test that simulates an existing KMS key Secret missing
the provider config to ensure the comparison path doesn't panic and the
controller recreates the key with the "secrets-kms-provider-changed"
internal-reason; copy one of the KMS table tests (e.g., the "creates a new KMS
key when VaultAddress changes" case), but construct initialObjects using
encryptiontesting.CreateMigratedEncryptionKeySecretWithKMSConfig or a similar
helper modified to produce a Secret that lacks the
"encryption.apiserver.operator.openshift.io-kms-provider-config" data key (or
contains malformed JSON), set apiServerObjects using simpleAPIServer with
dummyKMSConfig changes, and assert in validateFunc that a create:secrets action
occurs and the created Secret has annotation
encryption.apiserver.operator.openshift.io/internal-reason ==
"secrets-kms-provider-changed" and contains a valid provider config (no panic).
pkg/operator/encryption/kms/helpers_test.go (1)

33-117: ⚡ Quick win

Add coverage for nil and provider-type change branches.

Current table covers Vault field deltas well, but it misses latest/current == nil and Type mismatch paths, which are central guard rails for this helper.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pkg/operator/encryption/kms/helpers_test.go` around lines 33 - 117, Add two
table-driven test cases to the existing tests slice in helpers_test.go to cover
nil and provider-type-change branches: one case where latest is nil and current
is a non-nil *configv1.KMSConfig (expect true/indicating change) and one case
where latest is non-nil and current is nil (expect true), plus a case where both
are non-nil but their Type fields differ (e.g., latest.Type = "Vault" and
current.Type = "KMSPlugin") expecting true; reference the same KMSConfig/Vault
structures used in the file and ensure the cases follow the same shape (name,
latest, current, expected) as the other table entries so the test invokes the
same comparison helper being exercised by the existing tests.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@pkg/operator/encryption/controllers/key_controller.go`:
- Around line 367-373: The code dereferences latestKey.KMSConfig.Provider when
currentMode == state.KMS; add a nil/empty guard for latestKey.KMSConfig and
latestKey.KMSConfig.Provider before calling
kms.NeedsNewKey(latestKey.KMSConfig.Provider, currentApiServerEncryption.KMS);
if either is nil/empty, treat it as a provider change and return the same
migration result (e.g., return latestKeyID, "kms-provider-changed", true);
otherwise call kms.NeedsNewKey as before. Ensure the check surrounds the call to
kms.NeedsNewKey in key_controller.go so you never dereference a nil KMSConfig or
Provider.

In `@pkg/operator/encryption/kms/helpers.go`:
- Around line 16-25: The NeedsNewKey function currently dereferences latest and
current without nil checks; update it to first guard against nil inputs (e.g.,
if latest == nil || current == nil return true) before comparing Type, and when
handling provider-specific logic (NeedsNewKey's Vault branch) ensure
latest.Vault and current.Vault are non-nil before accessing VaultAddress,
VaultNamespace, TransitMount, TransitKey — if either Vault pointer is nil, treat
that as a change (return true) otherwise compare the individual fields. This
ensures nil-safety for NeedsNewKey and avoids controller panic.

In `@test/e2e-encryption/encryption_test.go`:
- Around line 514-519: Replace the fixed time.Sleep(5 * time.Second) with a
bounded poll that asserts "no new key" by repeatedly calling waitForKeys (or
querying the key count via the same mechanism used by waitForKeys) until the
count remains stable for a short stabilization window or a global timeout is
reached; after the fakeApiServerClient.Patch call (and after require.NoError),
implement polling that samples the key count, waits a small interval, rechecks,
and only proceeds to waitForConditionStatus("Encrypted",
operatorv1.ConditionTrue) when the key count has not increased during the
stabilization period or the timeout elapses, failing the test on timeout.

---

Nitpick comments:
In `@pkg/operator/encryption/controllers/key_controller_test.go`:
- Around line 622-774: Add a regression test that simulates an existing KMS key
Secret missing the provider config to ensure the comparison path doesn't panic
and the controller recreates the key with the "secrets-kms-provider-changed"
internal-reason; copy one of the KMS table tests (e.g., the "creates a new KMS
key when VaultAddress changes" case), but construct initialObjects using
encryptiontesting.CreateMigratedEncryptionKeySecretWithKMSConfig or a similar
helper modified to produce a Secret that lacks the
"encryption.apiserver.operator.openshift.io-kms-provider-config" data key (or
contains malformed JSON), set apiServerObjects using simpleAPIServer with
dummyKMSConfig changes, and assert in validateFunc that a create:secrets action
occurs and the created Secret has annotation
encryption.apiserver.operator.openshift.io/internal-reason ==
"secrets-kms-provider-changed" and contains a valid provider config (no panic).

In `@pkg/operator/encryption/kms/helpers_test.go`:
- Around line 33-117: Add two table-driven test cases to the existing tests
slice in helpers_test.go to cover nil and provider-type-change branches: one
case where latest is nil and current is a non-nil *configv1.KMSConfig (expect
true/indicating change) and one case where latest is non-nil and current is nil
(expect true), plus a case where both are non-nil but their Type fields differ
(e.g., latest.Type = "Vault" and current.Type = "KMSPlugin") expecting true;
reference the same KMSConfig/Vault structures used in the file and ensure the
cases follow the same shape (name, latest, current, expected) as the other table
entries so the test invokes the same comparison helper being exercised by the
existing tests.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 58265021-9d56-4fab-bdb5-25c2e172ce2c

📥 Commits

Reviewing files that changed from the base of the PR and between d45f1f2 and 3f864e3.

📒 Files selected for processing (6)
  • pkg/operator/encryption/controllers/key_controller.go
  • pkg/operator/encryption/controllers/key_controller_test.go
  • pkg/operator/encryption/kms/helpers.go
  • pkg/operator/encryption/kms/helpers_test.go
  • pkg/operator/encryption/testing/helpers.go
  • test/e2e-encryption/encryption_test.go

Comment thread pkg/operator/encryption/controllers/key_controller.go
Comment on lines +16 to +25
func NeedsNewKey(latest, current *configv1.KMSConfig) bool {
if latest.Type != current.Type {
// TODO: Integrate this with pre-flight checker
return true
}
if latest.Type == configv1.VaultKMSProvider {
if latest.Vault.VaultAddress != current.Vault.VaultAddress ||
latest.Vault.VaultNamespace != current.Vault.VaultNamespace ||
latest.Vault.TransitMount != current.Vault.TransitMount ||
latest.Vault.TransitKey != current.Vault.TransitKey {
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot Apr 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Add nil-safety to NeedsNewKey before dereferencing provider fields.

At Line 17, latest/current are dereferenced unconditionally. A nil input will panic the controller path.

Suggested fix
 func NeedsNewKey(latest, current *configv1.KMSConfig) bool {
+	if latest == nil || current == nil {
+		return latest != current
+	}
 	if latest.Type != current.Type {
 		// TODO: Integrate this with pre-flight checker
 		return true
 	}
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
func NeedsNewKey(latest, current *configv1.KMSConfig) bool {
if latest.Type != current.Type {
// TODO: Integrate this with pre-flight checker
return true
}
if latest.Type == configv1.VaultKMSProvider {
if latest.Vault.VaultAddress != current.Vault.VaultAddress ||
latest.Vault.VaultNamespace != current.Vault.VaultNamespace ||
latest.Vault.TransitMount != current.Vault.TransitMount ||
latest.Vault.TransitKey != current.Vault.TransitKey {
func NeedsNewKey(latest, current *configv1.KMSConfig) bool {
if latest == nil || current == nil {
return latest != current
}
if latest.Type != current.Type {
// TODO: Integrate this with pre-flight checker
return true
}
if latest.Type == configv1.VaultKMSProvider {
if latest.Vault.VaultAddress != current.Vault.VaultAddress ||
latest.Vault.VaultNamespace != current.Vault.VaultNamespace ||
latest.Vault.TransitMount != current.Vault.TransitMount ||
latest.Vault.TransitKey != current.Vault.TransitKey {
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pkg/operator/encryption/kms/helpers.go` around lines 16 - 25, The NeedsNewKey
function currently dereferences latest and current without nil checks; update it
to first guard against nil inputs (e.g., if latest == nil || current == nil
return true) before comparing Type, and when handling provider-specific logic
(NeedsNewKey's Vault branch) ensure latest.Vault and current.Vault are non-nil
before accessing VaultAddress, VaultNamespace, TransitMount, TransitKey — if
either Vault pointer is nil, treat that as a change (return true) otherwise
compare the individual fields. This ensures nil-safety for NeedsNewKey and
avoids controller panic.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

KMSConfig can not be nil if encryption mode is KMS

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ardaguclu, understood — since NeedsNewKey is only invoked on the KMS code path and KMSConfig is guaranteed to be non-nil in that context, the nil guard is unnecessary. Thanks for the clarification!


📝 Learnings were identified and not saved (knowledge base disabled). Enable

Comment on lines +514 to +519
t.Logf("KMS non-migration change: only KMSPluginImage changes (no new key expected)")
_, err = fakeApiServerClient.Patch(ctx, "cluster", types.MergePatchType, []byte(`{"spec":{"encryption":{"type":"KMS","kms":{"type":"Vault","vault":{"kmsPluginImage":"registry.example.com/kms-plugin@sha256:0000000000000000000000000000000000000000000000000000000000000000","vaultAddress":"https://vault-new.example.com","authentication":{"type":"AppRole","appRole":{"secret":{"name":"vault-approle-secret"}}},"transitKey":"test-transit-key"}}}}}`), metav1.PatchOptions{})
require.NoError(t, err)
time.Sleep(5 * time.Second)
waitForKeys(12)
waitForConditionStatus("Encrypted", operatorv1.ConditionTrue)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Replace fixed sleep with a bounded poll for “no new key” assertion.

Line 517’s time.Sleep(5 * time.Second) makes this check timing-sensitive. Prefer polling key count stability within a timeout window.

Suggested direction
-	time.Sleep(5 * time.Second)
-	waitForKeys(12)
+	err = wait.PollUntilContextTimeout(ctx, 200*time.Millisecond, 10*time.Second, true, func(ctx context.Context) (bool, error) {
+		l, err := kubeClient.CoreV1().Secrets("openshift-config-managed").List(ctx, metav1.ListOptions{LabelSelector: keySecretsLabel})
+		if err != nil {
+			return false, err
+		}
+		return len(l.Items) == 12, nil
+	})
+	require.NoError(t, err)
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
t.Logf("KMS non-migration change: only KMSPluginImage changes (no new key expected)")
_, err = fakeApiServerClient.Patch(ctx, "cluster", types.MergePatchType, []byte(`{"spec":{"encryption":{"type":"KMS","kms":{"type":"Vault","vault":{"kmsPluginImage":"registry.example.com/kms-plugin@sha256:0000000000000000000000000000000000000000000000000000000000000000","vaultAddress":"https://vault-new.example.com","authentication":{"type":"AppRole","appRole":{"secret":{"name":"vault-approle-secret"}}},"transitKey":"test-transit-key"}}}}}`), metav1.PatchOptions{})
require.NoError(t, err)
time.Sleep(5 * time.Second)
waitForKeys(12)
waitForConditionStatus("Encrypted", operatorv1.ConditionTrue)
t.Logf("KMS non-migration change: only KMSPluginImage changes (no new key expected)")
_, err = fakeApiServerClient.Patch(ctx, "cluster", types.MergePatchType, []byte(`{"spec":{"encryption":{"type":"KMS","kms":{"type":"Vault","vault":{"kmsPluginImage":"registry.example.com/kms-plugin@sha256:0000000000000000000000000000000000000000000000000000000000000000","vaultAddress":"https://vault-new.example.com","authentication":{"type":"AppRole","appRole":{"secret":{"name":"vault-approle-secret"}}},"transitKey":"test-transit-key"}}}}}`), metav1.PatchOptions{})
require.NoError(t, err)
err = wait.PollUntilContextTimeout(ctx, 200*time.Millisecond, 10*time.Second, true, func(ctx context.Context) (bool, error) {
l, err := kubeClient.CoreV1().Secrets("openshift-config-managed").List(ctx, metav1.ListOptions{LabelSelector: keySecretsLabel})
if err != nil {
return false, err
}
return len(l.Items) == 12, nil
})
require.NoError(t, err)
waitForConditionStatus("Encrypted", operatorv1.ConditionTrue)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@test/e2e-encryption/encryption_test.go` around lines 514 - 519, Replace the
fixed time.Sleep(5 * time.Second) with a bounded poll that asserts "no new key"
by repeatedly calling waitForKeys (or querying the key count via the same
mechanism used by waitForKeys) until the count remains stable for a short
stabilization window or a global timeout is reached; after the
fakeApiServerClient.Patch call (and after require.NoError), implement polling
that samples the key count, waits a small interval, rechecks, and only proceeds
to waitForConditionStatus("Encrypted", operatorv1.ConditionTrue) when the key
count has not increased during the stabilization period or the timeout elapses,
failing the test on timeout.

@ardaguclu ardaguclu force-pushed the kms-to-kms branch 2 times, most recently from 4fdbb32 to 3c4eb34 Compare May 5, 2026 06:26
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
test/e2e-encryption/encryption_test.go (1)

555-557: ⚡ Quick win

Use encoding.DecodeKMSConfig instead of raw JSON unmarshal.

Line 556 bypasses the shared decode path used elsewhere in this test. Reusing encoding.DecodeKMSConfig keeps this assertion aligned with the actual serialization contract and avoids drift if encoding behavior changes.

Suggested change
-	var providerConfig13 configv1.KMSConfig
-	require.NoError(t, json.Unmarshal(kmsProviderConfigData13, &providerConfig13))
-	require.Equal(t, "https://vault-new.example.com", providerConfig13.Vault.VaultAddress)
-	require.Equal(t, "test-transit-key", providerConfig13.Vault.TransitKey)
+	providerConfig13, err := encoding.DecodeKMSConfig(kmsProviderConfigData13)
+	require.NoError(t, err)
+	require.Equal(t, "https://vault-new.example.com", providerConfig13.Vault.VaultAddress)
+	require.Equal(t, "test-transit-key", providerConfig13.Vault.TransitKey)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/e2e-encryption/encryption_test.go` around lines 555 - 557, Replace the
direct json.Unmarshal call on kmsProviderConfigData13 with the shared decoder:
call encoding.DecodeKMSConfig(kmsProviderConfigData13, &providerConfig13) (or
the appropriate signature) instead of json.Unmarshal; keep the variable
providerConfig13 and the subsequent assertion require.Equal(t,
"https://vault-new.example.com", providerConfig13.Vault.VaultAddress) unchanged
so the test uses the common serialization/decoding path.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@test/e2e-encryption/encryption_test.go`:
- Around line 555-557: Replace the direct json.Unmarshal call on
kmsProviderConfigData13 with the shared decoder: call
encoding.DecodeKMSConfig(kmsProviderConfigData13, &providerConfig13) (or the
appropriate signature) instead of json.Unmarshal; keep the variable
providerConfig13 and the subsequent assertion require.Equal(t,
"https://vault-new.example.com", providerConfig13.Vault.VaultAddress) unchanged
so the test uses the common serialization/decoding path.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 367b45f1-563c-4184-897d-fd59eebb3636

📥 Commits

Reviewing files that changed from the base of the PR and between 4fdbb32 and 3c4eb34.

📒 Files selected for processing (11)
  • pkg/operator/encryption/controllers/key_controller.go
  • pkg/operator/encryption/controllers/key_controller_test.go
  • pkg/operator/encryption/controllers/state_controller_test.go
  • pkg/operator/encryption/encryptiondata/config.go
  • pkg/operator/encryption/encryptiondata/secret.go
  • pkg/operator/encryption/kms/helpers.go
  • pkg/operator/encryption/kms/helpers_test.go
  • pkg/operator/encryption/secrets/secrets_test.go
  • pkg/operator/encryption/state/types.go
  • pkg/operator/encryption/testing/helpers.go
  • test/e2e-encryption/encryption_test.go
🚧 Files skipped from review as they are similar to previous changes (2)
  • pkg/operator/encryption/kms/helpers.go
  • pkg/operator/encryption/kms/helpers_test.go

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
test/e2e-encryption/encryption_test.go (1)

558-560: ⚡ Quick win

Use encoding.DecodeKMSConfig instead of raw json.Unmarshal for provider payloads.

At Line 559, decoding via the same helper used elsewhere keeps this test resilient if provider serialization details evolve.

Proposed change
-	var providerConfig13 configv1.KMSConfig
-	require.NoError(t, json.Unmarshal(kmsProviderConfigData13, &providerConfig13))
-	require.Equal(t, "https://vault-new.example.com", providerConfig13.Vault.VaultAddress)
-	require.Equal(t, "test-transit-key", providerConfig13.Vault.TransitKey)
+	providerConfig13, err := encoding.DecodeKMSConfig(kmsProviderConfigData13)
+	require.NoError(t, err)
+	require.Equal(t, "https://vault-new.example.com", providerConfig13.Vault.VaultAddress)
+	require.Equal(t, "test-transit-key", providerConfig13.Vault.TransitKey)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/e2e-encryption/encryption_test.go` around lines 558 - 560, Replace the
raw json.Unmarshal call with the shared helper encoding.DecodeKMSConfig to
decode provider payloads: instead of calling
json.Unmarshal(kmsProviderConfigData13, &providerConfig13) use
encoding.DecodeKMSConfig(kmsProviderConfigData13, &providerConfig13) and assert
no error (require.NoError on the DecodeKMSConfig call); keep the subsequent
require.Equal on providerConfig13.Vault.VaultAddress unchanged. This updates the
test to use the same decoding path as other tests and stays resilient to
provider serialization changes.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@test/e2e-encryption/encryption_test.go`:
- Around line 558-560: Replace the raw json.Unmarshal call with the shared
helper encoding.DecodeKMSConfig to decode provider payloads: instead of calling
json.Unmarshal(kmsProviderConfigData13, &providerConfig13) use
encoding.DecodeKMSConfig(kmsProviderConfigData13, &providerConfig13) and assert
no error (require.NoError on the DecodeKMSConfig call); keep the subsequent
require.Equal on providerConfig13.Vault.VaultAddress unchanged. This updates the
test to use the same decoding path as other tests and stays resilient to
provider serialization changes.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: cc4000d9-f95c-4399-b8d7-d62f35f54adb

📥 Commits

Reviewing files that changed from the base of the PR and between 3c4eb34 and 1a33e46.

📒 Files selected for processing (11)
  • pkg/operator/encryption/controllers/key_controller.go
  • pkg/operator/encryption/controllers/key_controller_test.go
  • pkg/operator/encryption/controllers/state_controller_test.go
  • pkg/operator/encryption/encryptiondata/config.go
  • pkg/operator/encryption/encryptiondata/secret.go
  • pkg/operator/encryption/kms/helpers.go
  • pkg/operator/encryption/kms/helpers_test.go
  • pkg/operator/encryption/secrets/secrets_test.go
  • pkg/operator/encryption/state/types.go
  • pkg/operator/encryption/testing/helpers.go
  • test/e2e-encryption/encryption_test.go
✅ Files skipped from review due to trivial changes (5)
  • pkg/operator/encryption/state/types.go
  • pkg/operator/encryption/encryptiondata/config.go
  • pkg/operator/encryption/controllers/key_controller.go
  • pkg/operator/encryption/secrets/secrets_test.go
  • pkg/operator/encryption/kms/helpers_test.go

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
pkg/operator/encryption/controllers/key_controller_test.go (1)

670-673: 💤 Low value

Minor: Error message says "encode" but operation is "decode".

The error message at line 672 says "failed to encode" but DecodeKMSConfig is being called. Same issue at line 715.

Suggested fix
 				kmsProviderConfigData := actualSecret.Data["encryption.apiserver.operator.openshift.io-kms-provider-config"]
 				providerConfig, err := encoding.DecodeKMSConfig(kmsProviderConfigData)
 				if err != nil {
-					ts.Fatalf("failed to encode KMS config: %v", err)
+					ts.Fatalf("failed to decode KMS config: %v", err)
 				}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/operator/encryption/controllers/key_controller_test.go` around lines 670
- 673, The test uses encoding.DecodeKMSConfig but the ts.Fatalf message
incorrectly says "failed to encode KMS config"; update the error messages for
the DecodeKMSConfig calls (the ts.Fatalf around providerConfig, err :=
encoding.DecodeKMSConfig(...)) to say "failed to decode KMS config: %v" so the
log matches the operation (repeat the same fix for the second DecodeKMSConfig
occurrence in the file).
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@pkg/operator/encryption/controllers/key_controller_test.go`:
- Around line 670-673: The test uses encoding.DecodeKMSConfig but the ts.Fatalf
message incorrectly says "failed to encode KMS config"; update the error
messages for the DecodeKMSConfig calls (the ts.Fatalf around providerConfig, err
:= encoding.DecodeKMSConfig(...)) to say "failed to decode KMS config: %v" so
the log matches the operation (repeat the same fix for the second
DecodeKMSConfig occurrence in the file).

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: e105136c-78f0-4611-8355-8d4e8ab5c4c0

📥 Commits

Reviewing files that changed from the base of the PR and between 1a33e46 and 118cd7e.

📒 Files selected for processing (5)
  • pkg/operator/encryption/controllers/key_controller.go
  • pkg/operator/encryption/controllers/key_controller_test.go
  • pkg/operator/encryption/kms/helpers.go
  • pkg/operator/encryption/kms/helpers_test.go
  • test/e2e-encryption/encryption_test.go
🚧 Files skipped from review as they are similar to previous changes (2)
  • pkg/operator/encryption/kms/helpers.go
  • pkg/operator/encryption/kms/helpers_test.go

@ardaguclu
Copy link
Copy Markdown
Member Author

/hold cancel

@openshift-ci openshift-ci Bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label May 6, 2026
@ardaguclu
Copy link
Copy Markdown
Member Author

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 6, 2026

@ardaguclu: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

// However, we need to create a new key if migration-triggering fields
// in the KMS provider configuration have changed.
if kms.NeedsNewKey(latestKey.KMSConfig.Provider, currentApiServerEncryption.KMS) {
return latestKeyID, "kms-provider-changed", true
Copy link
Copy Markdown
Contributor

@p0lyn0mial p0lyn0mial May 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how could we e2e test this change ? :)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll update kms plugin deployment in library-go with the fake Vault kms plugin. We'll have actual Vault kms plugin coming from stepregistry. We'll move from one to another :)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, there is simpler way: We have already run multiple kms plugins (km-1.sock, kms-2.sock, etc.). We'll just migrate from one to another.

// in a way that requires creating a new encryption key and migrating storage.
// Only fields that affect the Key Encryption Key (KEK) trigger migration.
// Fields like KMSPluginImage, TLS, and Authentication are non-migration fields.
func NeedsNewKey(latest, current *configv1.KMSConfig) bool {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think this could be internal to the key_controller.

won't be used by the preflight-checker.
the key_controller will drive the preflight-checker

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense. So NeedsNewKey will return true, key_controller interacts with pre-flight checker.

// in a way that requires creating a new encryption key and migrating storage.
// Only fields that affect the Key Encryption Key (KEK) trigger migration.
// Fields like KMSPluginImage, TLS, and Authentication are non-migration fields.
func NeedsNewKey(latest, current *configv1.KMSConfig) bool {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are we missing a nil check for both params?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants