Documentation for "Immutable Secrets/ConfigMaps" feature #19297

wojtek-t · 2020-02-25T07:58:06Z

Ref kubernetes/enhancements#1412

/assign @cheftako

k8sio-netlify-preview-bot · 2020-02-25T08:09:35Z

Deploy preview for kubernetes-io-vnext-staging failed.

Built with commit ddb7248

https://app.netlify.com/sites/kubernetes-io-vnext-staging/deploys/5e54d3903bf8960009a34189

k8sio-netlify-preview-bot · 2020-02-25T08:09:37Z

Deploy preview for kubernetes-io-vnext-staging processing.

Building with commit 68c52ff

https://app.netlify.com/sites/kubernetes-io-vnext-staging/deploys/5e6a445744df3600093e5d5b

sftim

Hi @wojtek-t

Thanks for suggesting this documentation. Here's some feedback.

/sig apps

sftim · 2020-02-26T16:55:00Z

content/en/docs/concepts/configuration/secret.md

+{{< feature-state for_k8s_version="v1.18" state="alpha" >}}
+
+For clusters that are extensively using secrets (having at least tens of thousands of unique
+secret to pod mounts), we designed the `Immutable Secrets/ConfigMaps` feature. It allows you


Immutable Secrets/ConfigMaps

For highlighting, I'd use:

Suggested change

secret to pod mounts), we designed the `Immutable Secrets/ConfigMaps` feature. It allows you

Secret to Pod mounts), Kubernetes has an _Immutable Secrets & ConfigMaps_ feature. It allows you

content/en/docs/concepts/configuration/secret.md

wojtek-t · 2020-02-27T08:37:56Z

/sig apps

This isn't sig apps - it's sig/storage with combination of sig/node

wojtek-t · 2020-02-27T08:38:07Z

/sig node
/sig storage

wojtek-t · 2020-02-27T08:38:20Z

/assign @saad-ali

content/en/docs/concepts/configuration/secret.md

sftim

Some feedback

sftim · 2020-02-29T17:20:16Z

content/en/docs/concepts/configuration/secret.md

+
+To use it, you need to enable `ImmutableEmphemeralVolumes`
+[feature gate](/docs/reference/command-line-tools-reference/feature-gates/) and then
+mark your Secret/ConfigMap by setting it `Immutable` field to true, e.g.:


nits:

Suggested change

mark your Secret/ConfigMap by setting it `Immutable` field to true, e.g.:

mark your Secret/ConfigMap by setting its `Immutable` field to true; for example:

sftim · 2020-02-29T17:20:24Z

content/en/docs/concepts/configuration/secret.md

+```
+
+Once marked as immutable, it is NOT possible to revert this change nor to mutate the contents
+of data field. You can only delete and recreate the secrete, but that will not be picked by


nit:

Suggested change

of data field. You can only delete and recreate the secrete, but that will not be picked by

of data field. You can only delete and recreate the Secret, but that will not be picked by

Also, I'd put this in a {{< note >}} or {{< caution >}} block to highlight the gotcha about Pod replacement.

VineethReddy02 · 2020-03-07T13:06:16Z

👋 Please rebase this PR on dev-1.18 in order to avoid merge conflicts.

Feel free to /hold cancel after you've rebased.

/hold

sftim

@wojtek-t remember also update https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/

I've spotted a few other details that I recommend addressing.

sftim · 2020-03-08T22:39:51Z

content/en/docs/concepts/configuration/secret.md

+For clusters that are extensively using secrets (having at least tens of thousands of unique
+Secret to Pod mounts), Kubernetes has an _Immutable Secrets/ConfigMaps_ feature. It allows you
+to mark individual Secrets/ConfigMaps as `Immutable`, which blocks any changes to their data
+and in return:


Can I use an immutable Secret if I only have a few hundred Secrets in my cluster?

Yes - but I don't see where the text suggests you can't.

For clusters that are extensively using secrets (having at least tens of thousands of unique Secret to Pod mounts), Kubernetes has an Immutable Secrets/ConfigMaps feature.

This could read like a requirement.

What do you suggest? [I don't read it as a requirement]

sftim · 2020-03-08T22:40:44Z

content/en/docs/concepts/configuration/secret.md

+
+To use it, you need to enable `ImmutableEmphemeralVolumes`
+[feature gate](/docs/reference/command-line-tools-reference/feature-gates/) and then
+mark your Secret/ConfigMap by setting its `Immutable` field to true, e.g.:


nit:

Suggested change

mark your Secret/ConfigMap by setting its `Immutable` field to true, e.g.:

mark your Secret or ConfigMap, by setting its `immutable` field to `true`;

for example:

wojtek-t · 2020-03-11T19:53:11Z

@kbhawkey - can you please take another pass (since we already have a technical lgtm now).

kbhawkey · 2020-03-11T20:07:34Z

👀

kbhawkey · 2020-03-11T20:12:44Z

Page previews:
https://deploy-preview-19297--kubernetes-io-vnext-staging.netlify.com/docs/concepts/configuration/secret/

https://deploy-preview-19297--kubernetes-io-vnext-staging.netlify.com/docs/reference/command-line-tools-reference/feature-gates/

kbhawkey · 2020-03-11T20:26:41Z

content/en/docs/reference/command-line-tools-reference/feature-gates.md

@@ -105,6 +105,7 @@ different Kubernetes components.
 | `HPAScaleToZero` | `false` | Alpha | 1.16 | |
 | `HugePageStorageMediumSize` | `false` | Alpha | 1.18 | |
 | `HyperVContainer` | `false` | Alpha | 1.10 | |
+| `ImmutableEphemeralVolumes` | `false` | Alpha | 1.18 | |


@wojtek-t , If time, you probably want to add a brief description about this feature.
See the feature-gates section on this page, https://deploy-preview-19297--kubernetes-io-vnext-staging.netlify.com/docs/reference/command-line-tools-reference/feature-gates/#feature-gates

kbhawkey · 2020-03-11T20:48:59Z

content/en/docs/concepts/configuration/secret.md

@@ -676,6 +676,35 @@ A container using a Secret as a
 Secret updates.
 {{< /note >}}

+{{< feature-state for_k8s_version="v1.18" state="alpha" >}}


@wojtek-t , Here are some suggestions.
Could you create a sub-heading to introduce this section?
Right now this content is under Using Secrets and mounted-secrets-are-updated-automatically.
Not sure if this makes sense?

#### Immutable Secrets and ConfigMaps

Possible edit to introduce feature:

The Kubernetes alpha feature Immutable Secrets and ConfigMaps provides an option to set individual Secrets and ConfigMaps as immutable. For clusters that extensively use Secrets (at least tens of thousands of unique Secret to Pod mounts), preventing changes to this data has the following advantages:

I actually think that having it exactly in this section (mounted-secrets-are-updated-automatically) is exactly where it should be, because those secrets immutable secrets are the ones that won't be updated.

That said, I like the suggestion you have to introduce the feature - changing.

kbhawkey · 2020-03-11T20:51:24Z

content/en/docs/concepts/configuration/secret.md

+and in return:
+- protects you from accidental (or unwanted) updates that could cause applications outages
+- improves performance of your cluster by significantly reducing load on kube-apiserver, by
+closing watches for secrets marked as immutable.


nit, Line 686, closing watches?

improves performance of your cluster by significantly reducing load on the kube-apiserver and closing watches for secrets marked as immutable.

I think that "by" is better - because the performance is improved by the fact that we're closing those watches.

kbhawkey · 2020-03-11T21:01:59Z

content/en/docs/concepts/configuration/secret.md

+- improves performance of your cluster by significantly reducing load on kube-apiserver, by
+closing watches for secrets marked as immutable.
+
+To use it, you need to enable `ImmutableEmphemeralVolumes`


Possible edit, something like:

To use this feature, enable the ImmutableEmphemeralVolumes feature gate
and set your Secret or ConfigMap immutable field to true. For example:

kbhawkey · 2020-03-11T21:19:38Z

content/en/docs/concepts/configuration/secret.md

+
+{{< note >}}
+Once marked as immutable, it is NOT possible to revert this change nor to mutate the contents
+of data field. You can only delete and recreate the Secret, but that will not be picked by


Possible edit, pick and choose what makes sense:
Once a Secret is marked as immutable, it is not possible to revert this change nor to mutate the contents of the data field. You can only delete and recreate the Secret. Existing Pods maintain a mount point to the deleted Secret. It is recommended to recreate these Pods.

wojtek-t

@kbhawkey - thanks a lot for comments. PTAL

wojtek-t · 2020-03-12T12:56:04Z

content/en/docs/concepts/configuration/secret.md

@@ -676,6 +676,35 @@ A container using a Secret as a
 Secret updates.
 {{< /note >}}

+{{< feature-state for_k8s_version="v1.18" state="alpha" >}}


I actually think that having it exactly in this section (mounted-secrets-are-updated-automatically) is exactly where it should be, because those secrets immutable secrets are the ones that won't be updated.

That said, I like the suggestion you have to introduce the feature - changing.

wojtek-t · 2020-03-12T13:02:14Z

content/en/docs/concepts/configuration/secret.md

+- improves performance of your cluster by significantly reducing load on kube-apiserver, by
+closing watches for secrets marked as immutable.
+
+To use it, you need to enable `ImmutableEmphemeralVolumes`


wojtek-t · 2020-03-12T13:02:16Z

content/en/docs/concepts/configuration/secret.md

+and in return:
+- protects you from accidental (or unwanted) updates that could cause applications outages
+- improves performance of your cluster by significantly reducing load on kube-apiserver, by
+closing watches for secrets marked as immutable.


I think that "by" is better - because the performance is improved by the fact that we're closing those watches.

wojtek-t · 2020-03-12T13:05:04Z

content/en/docs/concepts/configuration/secret.md

+
+{{< note >}}
+Once marked as immutable, it is NOT possible to revert this change nor to mutate the contents
+of data field. You can only delete and recreate the Secret, but that will not be picked by


wojtek-t · 2020-03-12T13:08:03Z

content/en/docs/reference/command-line-tools-reference/feature-gates.md

@@ -105,6 +105,7 @@ different Kubernetes components.
 | `HPAScaleToZero` | `false` | Alpha | 1.16 | |
 | `HugePageStorageMediumSize` | `false` | Alpha | 1.18 | |
 | `HyperVContainer` | `false` | Alpha | 1.10 | |
+| `ImmutableEphemeralVolumes` | `false` | Alpha | 1.18 | |


kbhawkey · 2020-03-12T14:07:18Z

content/en/docs/concepts/configuration/secret.md

+individual Secrets and ConfigMaps as immutable. For clusters that extensively use Secrets
+(at least tens of thousands of unique Secret to Pod mounts), preventing changes to their
+data has the following advantages:
+- protects you from accidental (or unwanted) updates that could cause applications outages


@wojtek-t , Changes look good. One issue:
Add a newline at line 685 before the first list item (the list is not rendering)
Thanks!

wojtek-t · 2020-03-12T14:17:14Z

@kbhawkey - done (hopefully will work this time)

kbhawkey · 2020-03-12T14:28:18Z

/lgtm

@VineethReddy02 , Is the dev-1.18 branch accepting commits?

VineethReddy02 · 2020-03-12T20:15:39Z

@kbhawkey yes! If the technical reviews are done, we can merge this into dev-1.18

wojtek-t · 2020-03-12T20:40:45Z

@saad-ali approved it yesterday (there were only wording changes requested by @kbhawkey after that, so I believe we're fine).

kbhawkey · 2020-03-12T20:46:52Z

👍

wojtek-t · 2020-03-13T13:08:33Z

@kbhawkey - can you approve it then?

kbhawkey · 2020-03-13T13:14:42Z

@wojtek-t , I assume from the changes that you have rebased this pull request a number of times (there was a mixup with the dev-1.18 branch early in the release). If so, @VineethReddy02, I am happy to approve this pull request or if you agree, you can as well.

wojtek-t · 2020-03-13T13:24:51Z

@wojtek-t , I assume from the changes that you have rebased this pull request a number of times (there was a mixup with the dev-1.18 branch early in the release). If so, @VineethReddy02, I am happy to approve this pull request or if you agree, you can as well.

Yeah - I think it's ready. I would feel better if you could approve it.

sftim · 2020-03-13T13:55:33Z

Based on #19297 (comment) and #19297 (comment)
/approve

k8s-ci-robot · 2020-03-13T13:55:57Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: saad-ali, sftim

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~content/en/OWNERS~~ [sftim]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

wojtek-t · 2020-03-13T14:10:42Z

Thanks!

k8s-ci-robot assigned cheftako Feb 25, 2020

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Feb 25, 2020

wojtek-t force-pushed the immutable_secrets_docs branch from ddb7248 to 7232727 Compare February 25, 2020 07:59

k8s-ci-robot requested review from kbarnard10 and mikedanese February 25, 2020 07:59

wojtek-t mentioned this pull request Feb 25, 2020

Immutable Secrets and ConfigMaps kubernetes/enhancements#1412

Closed

sftim reviewed Feb 26, 2020

View reviewed changes

k8s-ci-robot added the sig/apps Categorizes an issue or PR as relevant to SIG Apps. label Feb 26, 2020

k8s-ci-robot added sig/node Categorizes an issue or PR as relevant to SIG Node. sig/storage Categorizes an issue or PR as relevant to SIG Storage. labels Feb 27, 2020

k8s-ci-robot assigned saad-ali Feb 27, 2020

wojtek-t force-pushed the immutable_secrets_docs branch 2 times, most recently from 834d2ad to 1c22124 Compare February 27, 2020 13:13

kbhawkey reviewed Feb 27, 2020

View reviewed changes

content/en/docs/concepts/configuration/secret.md Show resolved Hide resolved

k8s-ci-robot requested a review from VineethReddy02 February 27, 2020 15:38

sftim reviewed Feb 29, 2020

View reviewed changes

wojtek-t force-pushed the immutable_secrets_docs branch from 1c22124 to 4495e3e Compare March 2, 2020 20:01

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 7, 2020

sftim requested changes Mar 8, 2020

View reviewed changes

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 11, 2020

kbhawkey reviewed Mar 11, 2020

View reviewed changes

wojtek-t force-pushed the immutable_secrets_docs branch from b47e2a3 to 0b370eb Compare March 12, 2020 13:08

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 12, 2020

wojtek-t commented Mar 12, 2020

View reviewed changes

kbhawkey reviewed Mar 12, 2020

View reviewed changes

Immutable secrets doc

68c52ff

wojtek-t force-pushed the immutable_secrets_docs branch from 0b370eb to 68c52ff Compare March 12, 2020 14:16

k8s-ci-robot assigned kbhawkey Mar 12, 2020

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 12, 2020

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 13, 2020

k8s-ci-robot merged commit d343728 into kubernetes:dev-1.18 Mar 13, 2020

	secret to pod mounts), we designed the `Immutable Secrets/ConfigMaps` feature. It allows you
	Secret to Pod mounts), Kubernetes has an _Immutable Secrets & ConfigMaps_ feature. It allows you

	mark your Secret/ConfigMap by setting it `Immutable` field to true, e.g.:
	mark your Secret/ConfigMap by setting its `Immutable` field to true; for example:

	of data field. You can only delete and recreate the secrete, but that will not be picked by
	of data field. You can only delete and recreate the Secret, but that will not be picked by

	mark your Secret/ConfigMap by setting its `Immutable` field to true, e.g.:
	mark your Secret or ConfigMap, by setting its `immutable` field to `true`;
	for example:

Documentation for "Immutable Secrets/ConfigMaps" feature #19297

Documentation for "Immutable Secrets/ConfigMaps" feature #19297

Conversation

wojtek-t commented Feb 25, 2020

k8sio-netlify-preview-bot commented Feb 25, 2020

k8sio-netlify-preview-bot commented Feb 25, 2020 • edited Loading

sftim left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wojtek-t commented Feb 27, 2020

wojtek-t commented Feb 27, 2020

wojtek-t commented Feb 27, 2020

sftim left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

VineethReddy02 commented Mar 7, 2020

sftim left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wojtek-t commented Mar 11, 2020

kbhawkey commented Mar 11, 2020

kbhawkey commented Mar 11, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kbhawkey Mar 11, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wojtek-t left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wojtek-t commented Mar 12, 2020

kbhawkey commented Mar 12, 2020

VineethReddy02 commented Mar 12, 2020

wojtek-t commented Mar 12, 2020

kbhawkey commented Mar 12, 2020

wojtek-t commented Mar 13, 2020

kbhawkey commented Mar 13, 2020

wojtek-t commented Mar 13, 2020

sftim commented Mar 13, 2020

k8s-ci-robot commented Mar 13, 2020

wojtek-t commented Mar 13, 2020

k8sio-netlify-preview-bot commented Feb 25, 2020 •

edited

Loading

kbhawkey Mar 11, 2020 •

edited

Loading