Robust VolumeManager reconstruction after kubelet restart

[jsafrane]

Enhancement Description

• One-line enhancement description (can be used as a release note): Robust VolumeManager reconstruction after kubelet restart.
• Kubernetes Enhancement Proposal: KEP-3756: Add volume reconstruction KEP #3763
• Discussion Link:
  • Orphaned pod found, but error not a directory occurred when trying to remove the volumes dir kubernetes#105536 (comment)
• Primary contact (assignee): @jsafrane
• Responsible SIGs: sig-storage
• Enhancement target (which target equals to which milestone):
  • Alpha release target (x.y): 1.26 (as part of https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/1710-selinux-relabeling)
  • Beta release target (x.y): 1.27
  • Stable release target (x.y): TBD
• Beta
  • KEP (k/enhancements) update PR(s):
    • KEP-3756: Add volume reconstruction KEP #3763
  • Code (k/k) update PR(s):
    • Split volume reconstruction refactoring from SELinuxMountReadWriteOncePod kubernetes#115268
    • Add metric for failed orphan pod cleanup kubernetes#115972
    • Add volume reconstruction metrics kubernetes#115965
  • Docs (k/website) update(s): Add docs for VolumeManager reconstruction after kubelet restart website#40038
• Stable
  • KEP (k/enhancements) update PR(s):
  • Code (k/k) update PR(s):
  • Docs (k/website) update(s):

Please keep this description up to date. This will help the Enhancement Team to track the evolution of the enhancement efficiently.

[jsafrane]

/sig storage

[jsafrane]

/label lead-opted-in
/milestone v1.27
/stage alpha

[npolshakova]

Hello @jsafrane 👋, 1.27 Enhancements team here.

Just checking in as we approach enhancements freeze on 18:00 PDT Thursday 9th February 2023.

This enhancement is targeting for stage alpha for 1.27 (correct me, if otherwise)

Here's where this enhancement currently stands:

• KEP readme using the latest template has been merged into the k/enhancements repo.
• KEP status is marked as implementable for latest-milestone: 1.27
• KEP readme has a updated detailed test plan section filled out
• KEP readme has up to date graduation criteria
• KEP has a production readiness review that has been completed and merged into k/enhancements.

It looks like #3763 will address most of these issues.

The status of this enhancement is marked as at risk. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

[jsafrane]

This enhancement is targeting for stage alpha for 1.27 (correct me, if otherwise)

We're targeting Beta directly, we had alpha as part of https://github.com/kubernetes/enhancements/tree/master/keps/sig-storage/1710-selinux-relabeling

And the KEP is merged, together with PRR approval, I think everything is fine for 1.27

[npolshakova]

Great! This enhancement meets all the requirements for being included in v1.27 and is now tracked for the release.

One thing to note, make sure to update the PRR section in the KEP README.

/stage beta
/remove-label tracked/no
/label tracked/yes

[k8s-ci-robot]

@npolshakova: Those labels are not set on the issue: tracked/no

In response to this:

Great! This enhancement meets all the requirements for being included in v1.27 and is now tracked for the release.

One thing to note, make sure to update the PRR section in the KEP README.

/stage beta
/remove-label tracked/no
/label tracked/yes

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[mickeyboxell]

Hi @jsafrane 👋, I’m reaching out from the 1.27 Release Docs team. This enhancement is marked as ‘Needs Docs’ for the 1.27 release.

Please follow the steps detailed in the documentation to open a PR against dev-1.27 branch in the k/website repo. This PR can be just a placeholder at this time, and must be created by March 16. For more information, please take a look at Documenting for a release to familiarize yourself with the documentation requirements for the release.

Please feel free to reach out with any questions. Thanks!

[npolshakova]

Hi @jsafrane,

Checking in as we approach 1.27 code freeze at 17:00 PDT on Tuesday 14th March 2023.

Please ensure the following items are completed:

• All PRs to the Kubernetes repo that are related to your enhancement are linked in the above issue description (for tracking purposes).
• All PRs are fully merged by the code freeze deadline.

For this enhancement, it looks like the following PRs are open and need to be merged before code freeze:

• Split volume reconstruction refactoring from SELinuxMountReadWriteOncePod kubernetes#115268

Please let me know if there are any other PRs in k/k I should be tracking for this KEP.
As always, we are here to help should questions come up. Thanks!

[jsafrane]

There are kubernetes/kubernetes#115972 and kubernetes/kubernetes#115965 that are part of this feature and were already merged.

[jsafrane]

Docs: kubernetes/website#40038