Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Mark "Immutable Secrets/ConfigMaps" KEP implementable #1397

Merged
merged 1 commit into from Dec 17, 2019
Merged

Mark "Immutable Secrets/ConfigMaps" KEP implementable #1397

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
48 changes: 36 additions & 12 deletions ...e/20191117-immutable-ephemeral-volumes.md → .../20191117-immutable-secrets-configmaps.md
View file
Open in desktop
Expand Up @@ -14,8 +14,8 @@ reviewers:
approvers:
- "@saad-ali"
creation-date: 2019-11-17
last-updated: 2019-11-22
status: provisional
last-updated: 2019-12-09
status: implementable
see-also:
replaces:
superseded-by:
Expand Down Expand Up @@ -128,11 +128,16 @@ field:
```

If set, the machinery in apiserver will reject any updates of the object
trying to change anything different than ObjectMetadata. Note that we
need to allow mutating ObjectMetadata to not break object lifecycle
(e.g. to avoid deadlock if Finalizers are set).
trying to change anything different than ObjectMetadata.

Based on the value of that field Kubelet will or will not:
Note that will NOT reject all updates of the object, as we need to allow
e.g. for mutating ObjectMetadata (to not break object lifecycle, e.g. by
introducing a deadlock if Finalizers are set) or to allow rotating
certificates used for encryption at rest. We will only reject requests
that are explicitly changing keys and/or values stored in Secrets and/or
ConfigMaps.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@derekwaynecarr - I rephrased this paragraph to make it explicit what we discussed yesterday during the SIG.
PTAL

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for clarification.


Based on the value of `Immutable` field Kubelet will or will not:
- start a watch (or periodic polling) of a given Secret/ConfigMap
- perform updates of files mounted to a Pod based on updates of
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I might be wrong, but is volume remount is triggered by pod update, not volume update? Does volume (secret) update will trigger pod update?
https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/volumemanager/cache/actual_state_of_world.go#L69

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - we do update them periodically:
https://github.com/kubernetes/kubernetes/blob/cfdfd043a086eba12a673faf7a7dee89d28c82e3/pkg/volume/secret/secret.go#L85

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Secret volume (and configmap volume) is mounted read-only in the container, so users cannot update the volume contents.

The fact that the markremount function is in asw is wrong and it should be refactored eventually to dsw: kubernetes/kubernetes#77758

the Kubernetes object
Expand All @@ -150,27 +155,46 @@ the new logic behind the feature gate.

### Test Plan

**Note:** *Section not required until targeted at a release.*
For `Alpha`, we will add e2e tests verifying that contents of Secrets and
ConfigMaps marked as immutable really can't be updated. Additionally, these
will check if the metadata can be modified.

Additionally, unit tests will be added in Kubelet codebase to ensure that
the newly added logic to not watch immutable Secrets/ConfigMaps works as
expected.

TODO
For `Beta`, we will also extend scalability tests with a number of immutable
`Secrets` and `ConfigMaps` to validate the performance impact (for `Alpha`
only manual scalability tests will be performed).

### Graduation Criteria

**Note:** *Section not required until targeted at a release.*
Alpha:
- All tests describe above for `Alpha` are implemented and passing.
- Manual scalability tests prove the expected impact.

Beta:
- Scalability tests are extended to mount an immutable Secret and ConfigMap
for every single Pod, and that doesn't violate existing SLOs.

TODO
GA:
- No complaints about the API and user bug reports for 2 releases.

### Upgrade / Downgrade Strategy

No upgrade/downgrade concerns.
No upgrade/downgrade concerns.

### Version Skew Strategy

TODO
On Nodes in versions on supporting the feature, Kubelet will still be watching
immutable Secrets and/or ConfigMaps. That said, this is purely a performance
improvement and doesn't have correctness implications. So those clusters will
simple have worse scalability characteristic.

## Implementation History

2019-11-18: KEP opened
2019-12-09: KEP marked implementable

## Alternatives

Expand Down