Fix SELinux label of bootstrap-secrets on non-bootstrapping controllers #808
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dghubble
reviewed
Aug 19, 2020
CHANGES.md
Outdated
| @@ -33,6 +33,8 @@ Notable changes between versions. | |||
|
|
|||
| ### Fedora CoreOS | |||
|
|
|||
| * Fix SELinux label of bootstrap-secrets on non-bootstrapping controllers ([#808](https://github.com/poseidon/typhoon/pull/808)) | |||
There was a problem hiding this comment.
can you move to latest, this section is for v1.18.8
There was a problem hiding this comment.
@dghubble I thought v1.18.8 is on the same level with Fedora CoreOS since they are both prefixed with ###. It turns out that v1.18.8 should be prefixed with ##
|
I realized I only tested on v1.18.6. So I launched another testing cluster on v1.18.8 and found additional type label needs to be fixed as well. On the controllers that don't run This has been fixed in the latest commit. |
|
This is similar to the race in #708, but more limited. The label has a chance of appearing on non-bootstrap controllers in multi-controller clusters on the first boot from disk. Its also resolved on any restart of the kubelet.service (service, reboot, auto-update) for the duration of a cluster lifecycle. The randomness of appearance stems from the same race with the first kubelet start. Manually setting the context like this should eliminate its chance of happening during this window. Thanks |
Snaipe
pushed a commit
to aristanetworks/monsoon
that referenced
this pull request
Apr 13, 2023
…roller (poseidon#808) * Fix race condition for bootstrap-secrets SELinux context on non-bootstrap controllers in multi-controller FCOS clusters * On first boot from disk on non-bootstrap controllers, adding bootstrap-secrets races with kubelet.service starting, which can cause the secrets assets to have the wrong label until kubelet.service restarts (service, reboot, auto-update) * This can manifest as `kube-apiserver`, `kube-controller-manager`, and `kube-scheduler` pods crashlooping on spare controllers on first cluster creation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There's already a merged PR addressing the same problem on the controller that runs
bootstrapservice. However, on other controllers, Docker podskube-apiserver,kube-scheduler, andkube-controller-managerstill cannot read files from/etc/kubernetes/bootstrap-secrets(on host) due to the wrong SELinux file label (specificallyuserfield).On the controller that has run
bootstrapservice:On the controllers that don't run
bootstrapservice:See the difference in
userfield (system_uvsunconfined_u). This PR updates the user field tosystem_uon all controllers. Onaws/fedora-coreos, I've verified this fixes all the permission denials on files under/etc/kubernetes/secrets(container) fromkube-apiserver,kube-scheduler, andkube-controller-managercontainers.