Documentation for volume scheduling alpha feature #6392

msau42 · 2017-11-21T00:42:49Z

Need to document:

volume scheduling concept
new StorageClass VolumeBindingMode
update local volume documentation

This change is

k8sio-netlify-preview-bot · 2017-11-21T00:44:57Z

Deploy preview ready!

Built with commit 809604a

https://deploy-preview-6392--kubernetes-io-vnext-staging.netlify.com

msau42 · 2017-11-22T20:34:33Z

@tengqm @zacharysarah I noticed that StorageClasses got split out into its own page on the master branch. Are there plans to also cherry pick those changes to the 1.9 branch?

/assign @saad-ali @jsafrane @thockin

zacharysarah · 2017-11-22T20:41:27Z

@msau42 Yes; they'll get picked up the next time I merge master into release-1.9. In fact, I'll go do that right now.

tengqm · 2017-11-28T01:39:39Z

docs/concepts/storage/volumes.md

-This volume type is alpha in 1.7.
+{% assign for_k8s_version="v1.7" %}{% include feature-state-alpha.md %}
+
+This alpha feature requires the `PersistentLocalVolumes` feature gate to be


PersistentLocalVolumes became a thing since v1.9, how come the 'local' volume type is available since v1.7?

PersistentLocalVolumes was a new feature added in 1.7.

VolumeScheduling is a new feature in 1.9.

They are considered two separate features because the VolumeScheduling feature can theoretically work for more than just local volumes.

Hm this is even messier though. Starting in 1.9, the VolumeScheduling feature gate is required if you also need the PersistentLocalVolumes feature.

Okay, line 442 is clarifying this requirement.

Sorry can you clarify? I added a note stating that starting in 1.9, the VolumeScheduling feature gate is also required

Just did a search over the source code. It looks to me that the VolumeScheuling feature gate has to be enabled on the following components:

apiserver: for validation

kcm: for pv controller

scheduler: for scheduler behavior changes

For PersistentLocalVolumes, it has to be enabled on the following components:

apiserver: for validation

kubelet: for mounter

scheduler: for node affinity checking?

Yes that's correct. But like I mentioned in the other PR, should we just recommend that that the feature gates be set on all components? It would be a pain for an administrator to keep track of which feature gates are enabled where for each feature.

davidz627 · 2017-11-29T18:53:25Z

docs/concepts/storage/storage-classes.md

+### Volume Binding Mode
+{% assign for_k8s_version="v1.9" %}{% include feature-state-alpha.md %}
+
+This field requires the `VolumeScheduling` feature gate to be enabled.


Link for how to enable?

I don't think we have a page describing how to enable feature gates, but if you search the site, it will quickly point to some generated docs on each of the Kubernetes components and the flags that they take.

It might be beneficial to provide a link here so users don't have to search for it themselves.

Yeah, the challenge is there isn't just one link that describes the whole sequence. It's just every single kubernetes component has a help page describing their flags that you can pass.

davidz627 · 2017-11-29T18:55:53Z

docs/concepts/storage/volumes.md

@@ -481,6 +486,10 @@ spec:
 **Note:** The local PersistentVolume cleanup and deletion requires manual intervention without the external provisioner.
 {: .note}

+Starting in 1.9, local volume binding can be delayed by creating a StorageClass


"can be delayed [until scheduling time]" ?

verult · 2017-11-29T21:24:17Z

docs/concepts/storage/storage-classes.md

+
+A new field, `volumeBindingMode`, controls if volume binding should happen
+immediately or wait until pod scheduling.  Valid values are `Immediate` and
+`WaitForFirstConsumer`.  The default mode is `Immediate`.


Consider clarifying that Immediate is equivalent to the behavior prior to 1.9.

tengqm · 2017-11-30T06:45:57Z

docs/concepts/storage/persistent-volumes.md

+This requires the `VolumeScheduling` feature gate to be enabled.
+
+Normally, volume binding occurs immediately upon PVC creation.
+However, for PVs that specify NodeAffinity, it may be beneficial to delay


What is "PVs that specify NodeAffinity"?
From PersistentVolumeSpec or PersistentVolumeClaimSpec, I cannot find a hint where to specify node affinity.

Trying to make a guess ... it is about "PVs which require a node affinity be specified on the consuming Pod"?
Not accurate though. "PV types which are meant to be consumed on some specific nodes, where the nodes can be either specified using node affinity or can be determined after the pod is scheduled. In the latter case, a support to late binding is necessary."
Also, this seems only affecting dynamically provisioned volumes ...
please consider make this clear.

tengqm · 2017-11-30T07:02:57Z

docs/concepts/storage/persistent-volumes.md

+controlled through the `VolumeBindingMode` parameter in the
+[StorageClass](storage-classes.md#thestorageclassresource).
+
+**Note:** Dynamic provisioning is not supported yet with this alpha feature.


This is very confusing. If dynamic provisioning is not supported, what are the use cases of setting 'volumeBindingMode` property?

tengqm · 2017-11-30T07:03:47Z

docs/concepts/storage/persistent-volumes.md

+that the volume binding decision will also be evaluated with any other node
+constraints the pod may have, such as node resource requirements, node
+selectors, pod affinity, and pod anti-affinity.  This new mode of binding is
+controlled through the `VolumeBindingMode` parameter in the


from user's perspective, this is better described as "the volumeBindingMode field" for better matching the property name users see.

tengqm · 2017-11-30T07:05:29Z

docs/concepts/storage/storage-classes.md

+
+**Note:** The `WaitForFirstConsumer` volume binding mode does not support
+dynamic provisioning yet.  Also, it currently is only useful for the
+[local](volumes.md#local) volume type.  Additional volume types will be


ah ... I see. This is the only use case. Makes sense now .

I'd suggest we avoid document "predictions" about future, so the last sentence can be removed?

Since this feature is currently only useful for local volumes right now, I could move all of this under the local volume section instead. And when we support more volume types in the future, then I can move it out.

msau42 · 2017-12-01T20:29:07Z

Updated to only document the new changes under the local volume section, since it's currently restricted to only local volumes in the code.

msau42 · 2017-12-01T20:30:30Z

PTAL @jsafrane @saad-ali

zacharysarah

Good work. ✨ Minor formatting edits only:

Only one space after periods
Add note formatting to line 442

zacharysarah · 2017-12-01T22:48:28Z

docs/concepts/storage/persistent-volumes.md

@@ -60,7 +60,7 @@ please check [kube-apiserver](/docs/admin/kube-apiserver/) documentation.

 ### Binding

-A user creates, or has already created in the case of dynamic provisioning, a `PersistentVolumeClaim` with a specific amount of storage requested and with certain access modes.  A control loop in the master watches for new PVCs, finds a matching PV (if possible), and binds them together.  If a PV was dynamically provisioned for a new PVC, the loop will always bind that PV to the PVC. Otherwise, the user will always get at least what they asked for, but the volume may be in excess of what was requested.  Once bound, `PersistentVolumeClaim` binds are exclusive, regardless of the mode used to bind them.
+A user creates, or has already created in the case of dynamic provisioning, a `PersistentVolumeClaim` with a specific amount of storage requested and with certain access modes.  A control loop in the master watches for new PVCs, finds a matching PV (if possible), and binds them together.  If a PV was dynamically provisioned for a new PVC, the loop will always bind that PV to the PVC. Otherwise, the user will always get at least what they asked for, but the volume may be in excess of what was requested.  Once bound, `PersistentVolumeClaim` binds are exclusive, regardless of how they were bound.  A PVC to PV binding is a one-to-one mapping.


Only one space after periods, please.

done. I also fixed a bunch of other places nearby, so this change just got a little bigger.

zacharysarah · 2017-12-01T22:50:09Z

docs/concepts/storage/volumes.md

+This alpha feature requires the `PersistentLocalVolumes` feature gate to be
+enabled.
+
+Starting in 1.9, the `VolumeScheduling` feature gate must also be enabled.


This seems worth a note:

**Note:** Starting in 1.9, the `VolumeScheduling` feature gate must also be enabled. {: .note}

zacharysarah · 2017-12-01T23:14:19Z

Docs LGTM pending tech review 👍

jsafrane · 2017-12-02T09:28:13Z

lgtm from tech side

k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Nov 21, 2017

k8s-github-robot assigned jesscodez Nov 21, 2017

msau42 force-pushed the release-1.9 branch from a4b3ae3 to 5a86917 Compare November 22, 2017 20:24

k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Nov 22, 2017

k8s-ci-robot assigned jsafrane, saad-ali and thockin Nov 22, 2017

msau42 force-pushed the release-1.9 branch 2 times, most recently from b6ba54a to 937424f Compare November 22, 2017 23:42

msau42 changed the title ~~WIP Documentation for volume scheduling alpha feature~~ Documentation for volume scheduling alpha feature Nov 23, 2017

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 23, 2017

zacharysarah added this to the 1.9 milestone Nov 27, 2017

msau42 force-pushed the release-1.9 branch 3 times, most recently from 99ad3d2 to 3b83d07 Compare November 27, 2017 23:18

steveperry-53 added Needs Docs Review labels Nov 28, 2017

tengqm reviewed Nov 28, 2017

View reviewed changes

msau42 force-pushed the release-1.9 branch 2 times, most recently from 1ff6131 to 57ace35 Compare November 29, 2017 00:48

davidz627 reviewed Nov 29, 2017

View reviewed changes

steveperry-53 added Tech Review: Open Issues and removed Needs Tech Review labels Nov 29, 2017

verult reviewed Nov 29, 2017

View reviewed changes

msau42 force-pushed the release-1.9 branch from 57ace35 to 0c63121 Compare November 29, 2017 23:38

tengqm reviewed Nov 30, 2017

View reviewed changes

zacharysarah mentioned this pull request Nov 30, 2017

Release 1.9 #5978

Merged

msau42 force-pushed the release-1.9 branch from 0c63121 to 5fb3d4d Compare December 1, 2017 20:27

zacharysarah suggested changes Dec 1, 2017

View reviewed changes

zacharysarah added Docs Review: Open Issues and removed Needs Docs Review labels Dec 1, 2017

Documentation for volume scheduling alpha feature

809604a

msau42 force-pushed the release-1.9 branch from 5fb3d4d to 809604a Compare December 1, 2017 22:59

zacharysarah added Docs LGTM and removed Docs Review: Open Issues labels Dec 1, 2017

jsafrane added Tech Review LGTM and removed Tech Review: Open Issues labels Dec 2, 2017

tengqm merged commit 39908ed into kubernetes:release-1.9 Dec 2, 2017

zacharysarah mentioned this pull request Dec 7, 2017

[1.9] Issue Burndown kubernetes/sig-release#38

Closed

msau42 deleted the release-1.9 branch February 9, 2018 01:34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Documentation for volume scheduling alpha feature #6392

Documentation for volume scheduling alpha feature #6392

msau42 commented Nov 21, 2017 •

edited by k8s-reviewable

Loading

k8sio-netlify-preview-bot commented Nov 21, 2017 •

edited

Loading

msau42 commented Nov 22, 2017

zacharysarah commented Nov 22, 2017

tengqm Nov 28, 2017

msau42 Nov 28, 2017

msau42 Nov 28, 2017

msau42 Nov 28, 2017

tengqm Nov 30, 2017 •

edited

Loading

msau42 Nov 30, 2017

tengqm Nov 30, 2017

msau42 Nov 30, 2017

davidz627 Nov 29, 2017

msau42 Nov 29, 2017

davidz627 Nov 30, 2017

msau42 Nov 30, 2017

tengqm Nov 30, 2017

davidz627 Nov 29, 2017

msau42 Nov 29, 2017

verult Nov 29, 2017

msau42 Nov 29, 2017

tengqm Nov 30, 2017

tengqm Nov 30, 2017

tengqm Nov 30, 2017

tengqm Nov 30, 2017

msau42 Nov 30, 2017

msau42 commented Dec 1, 2017

msau42 commented Dec 1, 2017

zacharysarah left a comment

zacharysarah Dec 1, 2017

msau42 Dec 1, 2017

zacharysarah Dec 1, 2017

msau42 Dec 1, 2017

zacharysarah commented Dec 1, 2017

jsafrane commented Dec 2, 2017

Documentation for volume scheduling alpha feature #6392

Documentation for volume scheduling alpha feature #6392

Conversation

msau42 commented Nov 21, 2017 • edited by k8s-reviewable Loading

k8sio-netlify-preview-bot commented Nov 21, 2017 • edited Loading

msau42 commented Nov 22, 2017

zacharysarah commented Nov 22, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tengqm Nov 30, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

msau42 commented Dec 1, 2017

msau42 commented Dec 1, 2017

zacharysarah left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zacharysarah commented Dec 1, 2017

jsafrane commented Dec 2, 2017

msau42 commented Nov 21, 2017 •

edited by k8s-reviewable

Loading

k8sio-netlify-preview-bot commented Nov 21, 2017 •

edited

Loading

tengqm Nov 30, 2017 •

edited

Loading