KMS v2 Improvements

[ritazh]

Enhancement Description

• One-line enhancement description (can be used as a release note): Introduce KMS v2alpha1 API to add performance, rotation, and observability improvements
• Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/tree/master/keps/sig-auth/3299-kms-v2-improvements
• Discussion Link:
  • 3299-kms-v2-improvements #3302
  • Sig Auth meetings for 20220608: https://youtu.be/SNnvZVvk5VQ
• Primary contact (assignee): @ritazh @aramase
• Responsible SIGs: sig-auth
• Enhancement target (which target equals to which milestone):
  • Alpha release target (x.y): 1.25
  • Beta release target (x.y): 1.27
  • Stable release target (x.y): 1.29
• Alpha
  • KEP (k/enhancements) update PR(s):
    • 3299-kms-v2-improvements #3302
  • Code (k/k) update PR(s):
    • Implement KMS v2alpha1 kubernetes#111126
  • Docs (k/website) update PR(s):
    • KMS v2alpha1 website#35046
• Beta
  • KEP (k/enhancements) update PR(s):
    • KEP-3299: Update kmsv2 kep for beta #3804
  • Code (k/k) update PR(s):
    • kmsv2: implement expire cache with clock  kubernetes#113121
    • [KMSv2] Use status key ID to determine staleness of encrypted data kubernetes#114544
    • kmsv2: add grpc service kubernetes#114678
    • [KMSv2] apiserver/kmsv2: mv Service interface into kmsv2 kubernetes#114922
    • [KMSv2] Generate proto API and update feature gate for beta kubernetes#115123
    • [KMSv2] store hash of encrypted DEK as key in cache kubernetes#115350
    • kmsv2: add metrics kubernetes#115394
    • [KMSv2] Add metrics for grpc service kubernetes#115649
    • [KMSv2] implement local KEK service kubernetes#115677
    • [KMSv2] Add kind cluster and encryption config for e2e kubernetes#115714
    • [KMSv2] Implement local KEK generation and rotation kubernetes#115814
    • kmsv2: add metrics for invalid_key_id_from_status_total kubernetes#115846
    • [KMSv2] restructure kms staging dir kubernetes#115938
    • [KMSv2] update kms_operations_latency_seconds metric bucket range kubernetes#115947
    • [kmsv2] feat: add kms mock plugin for e2e tests kubernetes#116022
    • kmsv2: add mock kms for reference implementation kubernetes#116031
    • [KMSv2] remove setting dek_cache_inter_arrival_time_seconds for KMSv2 only kubernetes#116053
    • [KMSv2] log request metadata as part of read/write kubernetes#116055
    • [KMSv2] update ci script to create cluster and gather metrics kubernetes#116148
    • kmsv2: re-use DEK while key ID is unchanged kubernetes#116155
    • kmsv2: improve test coverage kubernetes#116202
    • [KMSv2] use encDEK, keyID and annotations to generate cache key kubernetes#116345
    • [KMSv2] fix: increases timeout to avoid flake kubernetes#116626
    • [KMSv2] remove key hierarchy in reference implementation kubernetes#116630
  • Docs (k/website) update PR(s):
    • Add docs to accompany KMS v2beta1 changes website#39110

[ritazh]

/sig auth

[ritazh]

/triage accepted

[enj]

/milestone v1.25

[Priyankasaggu11929]

Hello @ritazh 👋, 1.25 Enhancements team here.

Just checking in as we approach enhancements freeze on 18:00 PST on Thursday June 16, 2022.

For note, This enhancement is targeting for stage alpha for 1.25 (correct me, if otherwise)

Here's where this enhancement currently stands: (updated on June 9, 2022)

• KEP file using the latest template has been merged into the k/enhancements repo.
• KEP status is marked as implementable
• KEP has a updated detailed test plan section filled out
• KEP has up to date graduation criteria
• KEP has a production readiness review that has been completed and merged into k/enhancements.

Looks like for this one, we would need to update the following in the open KEP PR #3302:

• update the status in the kep.yaml file to implementable
• Update the KEP README.md file add updated detailed test plan section

For note, the status of this enhancement is marked as at risk. Thank you for keeping the issue description up-to-date!

[Priyankasaggu11929]

Hello @ritazh 👋, just a quick check-in again, as we approach the 1.25 enhancements freeze.

Please plan to get the pending items done before enhancements freeze on Thursday, June 23, 2022 at 18:00 PM PT.

For note, the current status of the enhancement is atat-risk. Thank you!

[Priyankasaggu11929]

With KEP PR #3302 merged, the enhancement is ready for 1.25 enhancements freeze.

For note, the status is now marked as tracked in the enhancements tracking sheet. Thank you!

[didicodes]

Hello @ritazh 👋, 1.25 Release Docs Shadow here.

This enhancement is marked as ‘Needs Docs’ for the 1.25 release. Please follow the steps detailed in the documentation to open a PR against the dev-1.25 branch in the k/website repo. This PR can be just a placeholder at this time and must be created by August 4.


Also, take a look at Documenting for a release to familiarize yourself with the docs requirement for the release. Thank you!

[Atharva-Shinde]

Hi @ritazh, Enhancements team here again 👋

Checking in as we approach Code Freeze at 01:00 UTC on Wednesday, 3rd August 2022.

Please ensure that the following items are completed before the code-freeze:

• All PRs to the Kubernetes repo that are related to your enhancement are linked in the above issue description (for tracking purposes).
• All PRs are fully merged by the code freeze deadline.

Currently, the status of the enhancement is marked as at-risk

Thanks :)

[Atharva-Shinde]

Hey @ritazh reaching out again as we approach Code Freeze at 01:00 UTC on this Wednesday i.e 3rd August 2022.
Try to get the action items done which are mentioned in the comment above before the code-freeze :)
The status of the enhancement is still marked as at-risk

[ritazh]

Thanks @Atharva-Shinde! kubernetes/kubernetes#111126 is currently waiting to be reviewed again by the approvers.

[Priyankasaggu11929]

Hello 👋, 1.25 Enhancements Lead here.

Unfortunately, this enhancement did not meet the code freeze criteria because there are still unmerged k/k code PRs.

If you still wish to progress this enhancement in v1.25, please file an exception request. Thank you so much!

/milestone clear

[ritazh]

@Priyankasaggu11929 FYI the exception request has been approved. kubernetes/kubernetes#111126 (comment)

[Priyankasaggu11929]

Thanks for the update @ritazh.

With the k/k PR also merged, I am marking it as tracked for 1.25 release cycle. Thank you so much!

[AdminTurnedDevOps]

Hey @ritazh

1.28 Docs Shadow here.

Does this enhancement work planned for 1.28 require any new docs or modification to existing docs?

If so, please follows the steps here to open a PR against dev-1.28 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday 20th July 2023.

Also, take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release.

Thank you!

[npolshakova]

Hey again @ritazh 👋

Just checking in as we approach Code freeze at 01:00 UTC Friday, 19th July 2023 .

Here’s the enhancement’s state for the upcoming code freeze:

• All the PRs that are related to your enhancement are linked in the above issue description (for tracking purposes). This includes code, tests, and documentation related PR/s.
• All code related PR/s are merged or are in merge-ready state ( i.e they have approved and lgtm labels applied) by the code freeze deadline. This includes any tests related PR/s too.
  • kmsv2: implement expire cache with clock  kubernetes#113121
  • [KMSv2] Use status key ID to determine staleness of encrypted data kubernetes#114544
  • kmsv2: add grpc service kubernetes#114678
  • [KMSv2] apiserver/kmsv2: mv Service interface into kmsv2 kubernetes#114922
  • [KMSv2] Generate proto API and update feature gate for beta kubernetes#115123
  • [KMSv2] store hash of encrypted DEK as key in cache kubernetes#115350
  • kmsv2: add metrics kubernetes#115394
  • [KMSv2] Add metrics for grpc service kubernetes#115649
  • [KMSv2] implement local KEK service kubernetes#115677
  • [KMSv2] Add kind cluster and encryption config for e2e kubernetes#115714
  • [KMSv2] Implement local KEK generation and rotation kubernetes#115814
  • kmsv2: add metrics for invalid_key_id_from_status_total kubernetes#115846
  • [KMSv2] restructure kms staging dir kubernetes#115938
  • [KMSv2] update kms_operations_latency_seconds metric bucket range kubernetes#115947
  • [kmsv2] feat: add kms mock plugin for e2e tests kubernetes#116022
  • kmsv2: add mock kms for reference implementation kubernetes#116031
  • [KMSv2] remove setting dek_cache_inter_arrival_time_seconds for KMSv2 only kubernetes#116053
  • [KMSv2] log request metadata as part of read/write kubernetes#116055
  • [KMSv2] update ci script to create cluster and gather metrics kubernetes#116148
  • kmsv2: re-use DEK while key ID is unchanged kubernetes#116155
  • kmsv2: improve test coverage kubernetes#116202
  • [KMSv2] use encDEK, keyID and annotations to generate cache key kubernetes#116345
  • [KMSv2] fix: increases timeout to avoid flake kubernetes#116626
  • [KMSv2] remove key hierarchy in reference implementation kubernetes#116630

Also please let me know if there are other PRs in k/k we should be tracking for this KEP.
As always, we are here to help if any questions come up. Thanks!

[Rishit-dagli]

Hey @ritazh , could you please create a docs PR even if it is a draft PR with no content yet against dev-1.28 branch in the k/website repo. The deadline to create this draft PR is Thursday 20th July 2023.

[aramase]

Hey @ritazh , could you please create a docs PR even if it is a draft PR with no content yet against dev-1.28 branch in the k/website repo. The deadline to create this draft PR is Thursday 20th July 2023.

Thanks! Opened kubernetes/website#42076.

[Atharva-Shinde]

Thanks for filling an exception request! I'll reapply the v1.28 milestone if the exception request is approved.
/milestone clear

The exception request was approved! Applying the v1.28 milestone again :)

[Rishit-dagli]

Hello @ritazh wave: please take a look at Documenting for a release - PR Ready for Review to get your docs PR ready for review before Tuesday 25th July 2023. Thank you!

Ref: kubernetes/website#42076

[npolshakova]

/remove-label lead-opted-in

[liggitt]

/label lead-opted-in

we're continuing work on this in 1.29, planning to graduate to GA

[sreeram-venkitesh]

Hello @ritazh 👋, Enhancements team here.

Just checking in as we approach enhancements freeze on 01:00 UTC, Friday, 6th October, 2023.

This enhancement is targeting for stage stable for v1.29 (correct me, if otherwise)

Here's where this enhancement currently stands:

• KEP readme using the latest template has been merged into the k/enhancements repo.
• KEP status is marked as implementable for latest-milestone: 1.29. KEPs targeting stable will need to be marked as implemented after code PRs are merged and the feature gates are removed.
• KEP readme has up-to-date graduation criteria
• KEP has a production readiness review that has been completed and merged into k/enhancements. (For more information on the PRR process, check here).

Everything is up to date and KEP is tracked for the enhancements freeze 🎉

The status of this enhancement is marked as tracked for enhancement freeze. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

[aramase]

Update the latest milestone to 1.29 in the kep.yaml file.

@sreeram-venkitesh The kep.yaml you have tagged is from my fork which is incorrect. The kep.yaml in the enhancements repo has latest milestone set to 1.29 for this KEP.

enhancements/keps/sig-auth/3299-kms-v2-improvements/kep.yaml

Line 18 in a2fb02e

latest-milestone: "v1.29"

Have a completed PRR review merged into k/enhancements for the stable stage.

Same issue as above. The link referenced/validated is from my fork. The PRR review is complete and the PRR in this repo is up-to-date.

enhancements/keps/prod-readiness/sig-auth/3299.yaml

Lines 6 to 7 in a2fb02e

	stable:
	approver: "@deads2k"

[sreeram-venkitesh]

@aramase Apologies for the oversight! Thanks for clarifying, I've updated my comment above and the enhancement tracking sheet to Tracked for enhancements freeze 🎉

[katcosgrove]

Hey there @enj and @ritazh! 👋, v1.29 Docs Lead here.
Does this enhancement work planned for v1.29 require any new docs or modification to existing docs?
If so, please follows the steps here to open a PR against dev-1.29 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday, 19 October 2023.
Also, take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release.
Thank you!

[James-Quigley]

Hi @ritazh 👋 from the v1.29 Communications Release Team! We would like to check if you have any plans to publish blogs for this KEP regarding new features, removals, and deprecations for this release.
If so, you need to open a PR placeholder in the website repository.
The deadline will be on Tuesday 14th November 2023 (after the Docs deadline PR ready for review)
Here's the 1.29 Calendar

[sreeram-venkitesh]

Hey again @ritazh 👋 v1.29 Enhancements team here,

Just checking in as we approach code freeze at 01:00 UTC Wednesday 1st November 2023 .

Here's where this enhancement currently stands:

• All PRs to the Kubernetes repo that are related to your enhancement are linked in the above issue description (for tracking purposes).
• All PR/s are ready to be merged (they have approved and lgtm labels applied) by the code freeze deadline. This includes tests.

The status of this KEP is currently at risk for Code Freeze. Please let me know if there are other PRs in k/k we should be tracking for this KEP.

As always, we are here to help if any questions come up. Thanks!

[sreeram-venkitesh]

Hello @ritazh @aramase 👋, 1.29 Enhancements team here.

With all the following implementation(code related) PRs merged into k/k, this enhancement is now marked as tracked for code freeze for the 1.29 Code Freeze! 🚀 Are there any other PRs other than the following ones? If there are any other PRs that needs to be tracked, please let us know. Also please update the issue description with the following PRs:

• [KMSv2] promote KMSv2 and KMSv2KDF to GA kubernetes#121485
• [KMSv2] Add tracing kubernetes#121095
• [KMSv2] pkcs11 reference implementation using SoftHSM kubernetes#120896
• kmsv2: add apiserver identity to metrics kubernetes#120438
• [KMSv2] Drop the apimachinery kubernetes#120225
• kmsv2: add metric for DEK cache filled kubernetes#119878

The test freeze is 01:00 UTC Wednesday 15th November 2023 / 18:00 PDT Tuesday 14th November 2023. Please make sure all test PRs are merged in by then. The tracked test PRs for this KEP are:

• kmsv2 test feature enablement unit test kubernetes#119714

Please let me know if there are additional test PRs we should track. Thanks!