Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Documenting FlexVolume Resize alpha feature. #10097

Merged
merged 1 commit into from
Nov 29, 2018

Conversation

brahmaroutu
Copy link
Contributor

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Please delete this note before submitting the pull request.
For 1.12 Features: set Milestone to 1.12 and Base Branch to release-1.12
Help editing and submitting pull requests: https://deploy-preview-9510--kubernetes-io-master-staging.netlify.com/docs/contribute/start/#submit-a-pull-request.
Help choosing which branch to use, see
https://kubernetes.io/docs/contribute/start#choose-which-git-branch-to-use.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

@aniket-s-kulkarni, @zparnold Please review.

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Aug 27, 2018
@k8s-ci-robot k8s-ci-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Aug 27, 2018
@k8sio-netlify-preview-bot
Copy link
Collaborator

k8sio-netlify-preview-bot commented Aug 27, 2018

Deploy preview for kubernetes-io-master-staging ready!

Built with commit 511f3a0

https://deploy-preview-10097--kubernetes-io-master-staging.netlify.com

@Bradamant3
Copy link
Contributor

@brahmaroutu can you please rebase against the 1.12-release branch? (See PR template for details.) Thanks!

@Bradamant3 Bradamant3 added this to the 1.12 milestone Aug 27, 2018
@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Aug 27, 2018
@brahmaroutu brahmaroutu changed the base branch from master to release-1.12 August 27, 2018 20:01
@k8s-ci-robot k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Aug 27, 2018
@zparnold zparnold modified the milestones: 1.12, 1.13 Sep 14, 2018
@zparnold zparnold changed the base branch from release-1.12 to master September 14, 2018 23:06
@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Sep 14, 2018
@tfogo tfogo changed the base branch from master to dev-1.13 October 12, 2018 23:38
@kubernetes-docs-i18n-bot kubernetes-docs-i18n-bot added the language/en Issues or PRs related to English language label Oct 12, 2018
@tfogo
Copy link
Contributor

tfogo commented Nov 19, 2018

Hi @brahmaroutu, could you please rebase on dev-1.13, fix any merge conflicts, and let us know when it's ready for review? The docs deadline is approaching soon so we want to start reviewing PRs ASAP.

@k8sio-netlify-preview-bot
Copy link
Collaborator

k8sio-netlify-preview-bot commented Nov 19, 2018

Deploy preview for kubernetes-io-vnext-staging processing.

Built with commit 1d992b5

https://app.netlify.com/sites/kubernetes-io-vnext-staging/deploys/5bfdba83e5cd166617999cb1

@k8s-ci-robot k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Nov 19, 2018
@msau42
Copy link
Member

msau42 commented Nov 19, 2018

/assign @gnufied

@@ -226,9 +226,22 @@ kubectl describe pvc <pvc_name>
```

If the `PersistentVolumeClaim` has the status `FileSystemResizePending`, it is safe to recreate the pod using the PersistentVolumeClaim.
{{< feature-state for_k8s_version="v1.12" state="alpha" >}}
FlexVolumes allow resize if the driver is set with the `RequiresFSResize` capability to true. Since this is a alpha feature
both `ExpandInUsePersistentVolumes` and `ExpandPersistentVolumes` feature gates must be enabled.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is alpha feature? Flex volume resizing is not feature gated. Flex volume does not need ExpandInUsePersistentVolumes for flex volume resizing. Lets not conflate these two. Also ExpandPersistentVolumes is already in beta and hence enabled by default.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gnufied reworded, please review.

@brahmaroutu brahmaroutu force-pushed the flexvolume_resize branch 2 times, most recently from beee72f to 681c668 Compare November 20, 2018 18:54
@k8s-ci-robot k8s-ci-robot added size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Nov 20, 2018
@tengqm
Copy link
Contributor

tengqm commented Nov 21, 2018

@brahmaroutu Please also update content/en/docs/reference/command-line-tools-reference/feature-gates.md where we consolidate all feature gates for users to reference. As gates come and go, it is very likely people will get confused.

@brahmaroutu
Copy link
Contributor Author

@tengqm The feature gates I mentioned in the documentation are already existing ones, I see them already listed in the content/en/docs/reference/command-line-tools-reference/feature-gates.md

@tengqm
Copy link
Contributor

tengqm commented Nov 22, 2018

@brahmaroutu You are right. Just confirmed that these feature gates have been updated in the dev-1.13 branch. Thanks.

The FlexVolume plugin uses RequiresFSResize to identify and resize the Persistent Volume. The underlying driver must support resize, if not an error will be returned and Persistent Volume object will not change and the expand request will fail. FlexVolume driver must implement the method `ExpandFS` defined in the interface `FSResizableVolumePlugin` along with `ExpandableVolumePlugin` interface and its method `ExpandVolumeDevice`.
The Flex driver should implement ExpandableVolumePlugin interface only if it is to be installed on the controller/master.
The Flex driver should implement any volume expansion methods along with FS resize in ExpandFS method if the Flex driver is installed on the worker nodes but not on the controller or master node.

{{< feature-state for_k8s_version="v1.11" state="alpha" >}}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This document is user facing piece. We do not document interfaces to implement here. Such details can be moved to - https://github.com/kubernetes/community/blob/master/contributors/devel/flexvolume.md

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A separate PR will be opened to move the text into the community page.

@@ -227,8 +227,17 @@ kubectl describe pvc <pvc_name>

If the `PersistentVolumeClaim` has the status `FileSystemResizePending`, it is safe to recreate the pod using the PersistentVolumeClaim.

FlexVolumes allow resize if the driver is set with the `RequiresFSResize` capability to true.
The FlexVolume can be resized on pod restart as 'ExpandPersistentVolume' feature gate is enabled by default.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no need to say that as 'ExpandPersistentVolume' feature gate is enabled by default. because it is already default.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the text about the feature gate.

The FlexVolume plugin uses RequiresFSResize to identify and resize the Persistent Volume. The underlying driver must support resize, if not an error will be returned and Persistent Volume object will not change and the expand request will fail. FlexVolume driver must implement the method `ExpandFS` defined in the interface `FSResizableVolumePlugin` along with `ExpandableVolumePlugin` interface and its method `ExpandVolumeDevice`.
The Flex driver should implement ExpandableVolumePlugin interface only if it is to be installed on the controller/master.
The Flex driver should implement any volume expansion methods along with FS resize in ExpandFS method if the Flex driver is installed on the worker nodes but not on the controller or master node.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO - we should delete this entire section. There isn't anything special about flex that needs documenting here. Flex specific implementation details should go in document I linked below.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's probably important to keep the sentence "The underlying driver must support resize" since that there's a chance administrators might be confused if this isn't working for the drivers they have deployed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

partly agree but at very least it should go below previous text. This section is about expanding in-use persistent volumes and not just flex volume, so there is no need to document this first thing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a not to satisfy this, please review.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. But what I meant was - Resizing an in-use PersistentVolumeClaim section introduces expansion of in-use volumes regardless of plugin type. This section is not just about flex. But currently you have put flex item as first thing under in-use section. So if you read rendered markdown, it reads weird.

Resizing an in-use PersistentVolumeClaim

Expanding in-use PVCs for FlexVolumes is added in release 1.13. To enable this feature use ExpandInUsePersistentVolumes and ExpandPersistentVolumes feature gates. The ExpandPersistentVolumes feature gate is already enabled by default. If the ExpandInUsePersistentVolumes is set, FlexVolume can be resized online without pod restart.

{{< note >}} Note: FlexVolume resize is possible only when the underlying driver supports resize. {{< /note >}}

{{< feature-state for_k8s_version="v1.11" state="alpha" >}}

Expanding in-use PVCs is an alpha feature. To use it, enable the ExpandInUsePersistentVolumes feature gate. In this case, you don't need to delete and recreate a Pod or deployment that is using an existing PVC. Any in-use PVC automatically becomes available to its Pod as soon as its file system has been expanded. This feature has no effect on PVCs that are not in use by a Pod or deployment. You must create a Pod which uses the PVC before the expansion can complete.

{{< note >}} Note: Expanding EBS volumes is a time consuming operation. Also, there is a per-volume quota of one modification every 6 hours. {{< /note >}}

This is how the whole text reads as. Whereas what I was suggesting is - this diff should not alter first paragraph of the section. The blurb about flexvolume can be added as second paragraph rather than sitting on top of the section. Hope that clarifies it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is how I propose the section should read:

Expanding in-use PVCs is an alpha feature. To use it, enable the ExpandInUsePersistentVolumes feature gate. In this case, you don't need to delete and recreate a Pod or deployment that is using an existing PVC. Any in-use PVC automatically becomes available to its Pod as soon as its file system has been expanded. This feature has no effect on PVCs that are not in use by a Pod or deployment. You must create a Pod which uses the PVC before the expansion can complete.

Expanding in-use PVCs for FlexVolumes is added in release 1.13. To enable this feature use ExpandInUsePersistentVolumes and ExpandPersistentVolumes feature gates. The ExpandPersistentVolumes feature gate is already enabled by default. If the ExpandInUsePersistentVolumes is set, FlexVolume can be resized online without pod restart.
{{< note >}} Note: FlexVolume resize is possible only when the underlying driver supports resize. {{< /note >}}

{{< feature-state for_k8s_version="v1.11" state="alpha" >}}

{{< note >}} Note: Expanding EBS volumes is a time consuming operation. Also, there is a per-volume quota of one modification every 6 hours. {{< /note >}}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it, thanks for the review.

@k8s-ci-robot k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Nov 27, 2018
@brahmaroutu
Copy link
Contributor Author

@gnufied hope I addressed all your review comments.

@gnufied
Copy link
Member

gnufied commented Nov 28, 2018

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Nov 28, 2018
@tfogo
Copy link
Contributor

tfogo commented Nov 29, 2018

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: tfogo

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 29, 2018
@k8s-ci-robot k8s-ci-robot merged commit fbbab9d into kubernetes:dev-1.13 Nov 29, 2018
k8s-ci-robot pushed a commit that referenced this pull request Dec 4, 2018
* Update metadata.generation behaviour for custom resources (#10705)

* update docs promoting plugins to beta (#10796)

* docs update to promote TaintBasedEvictions to beta (#10765)

* First Korean l10n work for dev-1.13 (#10719)

* Update outdated l10n(ko) contents (#10689)

fixes #10686

* Translate concepts/overview/what-is-kubernetes in Korean (#10690)

* Translate concepts/overview/what-is-kubernetes in Korean

* Feedback from ClaudiaJKang

* Translate concepts/overview/components in Korean (#10882)

* Translate concepts/overview/components in Korean #10717

* Translate concepts/overview/components in Korean

* Translate concepts/overview/components in Korean

* Apply Korean glossary: 서비스 어카운트

* Translate concepts/overview/kubernetes-api in Korean (#10773)

* Translate concepts/overview/kubernetes-api in Korean

* Applied feedback from ianychoi

* kubeadm: update the configuration docs to v1beta1 (#10959)

* kubeadm: add small v1beta1 related updates (#10988)

* ADD content/zh/docs/reference/setup-tools/kubeadm/kubeadm.md (#11031)

* ADD content/zh/docs/reference/setup-tools/kubeadm/kubeadm.md

* ADD content/zh/docs/reference/setup-tools/kubeadm/generated/kubeadm_init.md

* Update content/zh/docs/reference/setup-tools/kubeadm/kubeadm.md

Accepted

Co-Authored-By: YouthLab <tsui@highyouth.com>

* do not change 'master' or 'worker' nodes to '主从'

* Doc updates for volume scheduling GA (#10743)

* Doc updates for volume scheduling GA

* Make trivial change to kick build

* Document nodelease feature (#10699)

* advanced audit doc for ModeBlockingStrict (#10203)

* Rename EncryptionConfig to EncryptionConfiguration (#11080)

EncryptionConfig was renamed to EncryptedConfiguration and added to
the `apiserver.config.k8s.io` API group in Kubernetes 1.13.

The feature was previously in alpha and was not handling versions
properly, which lead to an originally unnoticed `v1` in the docs.

* content/zh/docs/reference/setup-tools/kubeadm/kubeadm-init.md

* trsanlate create-cluster-kubeadm.md to chinese (#11041)

* trsanlate create-cluster-kubeadm.md to chinese

* Update create-cluster-kubeadm.md

* update the feature stage in v1.13 (#11307)

* update new feature gates to document (#11295)

* refresh controller role list on rbac description page (#11290)

* node labeling restriction docs (#10944)

* Update 1.13 docs for CSI GA (#10893)

* dynamic audit documentation (#9947)

* adds dynamic audit documentation

* Copyedit for clarity

See also inline question/s

* Fix feature state shortcode

* Update feature state

* changes wording for dynamic audit flag behavior

* Minor copyedit

* fix dynamic audit yaml

* adds api enablement command to dynamic audit docs

* change ordering dynamic audit appears in

* add references to dynamic audit in webhook backend

* reword dynamic audit reference

* updates stages field for audit sink object

* changes audit sink api definition; rewords policy

* kubeadm: remove kube-proxy workaround (#11162)

* zh-trans content/en/docs/setup/independent/install-kubeadm.md (#11338)

* zh-trans content/en/docs/setup/independent/install-kubeadm.md

* Update install-kubeadm.md

* Update dry run feature to beta (#11140)

* vSphere volume raw block support doc update (#10932)

* Add docs for Windows DNS configurations (#10036)

* Update docs for fields allowed at root of CRD schema (#9973)

* Add docs for Windows DNS configurations

* add device monitoring documentation (#9945)

* kubeadm: adds upgrade instructions for 1.13 (#11138)

* kubeadm: adds upgrade instructions for 1.13

Signed-off-by: Chuck Ha <ha.chuck@gmail.com>

* add minor copyedits

Addressed a couple of copyedit comments a bit more cleanly.

* kubeadm: add improvements to HA docs (#11094)

* kubeadm: add information and diagrams for HA topologies

* kubeadm: update HA doc with simplified steps

* kubeadm: update HA doc with simplified steps

* edit ha, add new topology topic, reorder by weight

* troubleshoot markdown

* fix more markdown, fix links

* more markdown

* more markdown

* more markdown

* changes after reviewer comments

* add steps about Weave

* update note about stacked topology

* kubeadm external etcd HA upgrade 1.13 (#11364)

* kubeadm external etcd HA upgrade 1.13

Signed-off-by: Ruben Orduz <rubenoz@gmail.com>

* Update stacked controlplane steps

* kubeadm cert documentation (#11093)

* kubeadm certificate API and CSR documentation

* copyedits

* fix typo

* PR for diff docs (#10789)

* Empty commit against dev-1.13 for diff documentation

* Complete Declarative maangement with diff commands

* Second Korean l10n work for dev-1.13. (#11030)

* Update outdated l10n(ko) contents (#10915)
* Translate main menu for l10n(ko) docs (#10916)
* Translate tasks/run-application/horizontal-pod-autoscale-walkthrough (#10980)
* Translate content/ko/docs/concepts/overview/working-with-objects/kubernetes-object in Korean #11104 (#11332)
* Pick-right-solution page translates into Korean. (#11340)
* ko-trans: add jd/..., sap/..., ebay/..., homeoffice/... (#11336)
* Translate concept/workloads/pods/pod-overview.md (#11092)

Co-authored-by: June Yi <june.yi@samsung.com>
Co-authored-by: Jesang Myung <jesang.myung@gmail.com>
Co-authored-by: zerobig <38598117+zer0big@users.noreply.github.com>
Co-authored-by: Claudia J.Kang <claudiajkang@gmail.com>
Co-authored-by: lIuDuI <1693291525@qq.com>
Co-authored-by: Woojin Na(Eddie) <cheapluv@gmail.com>

* Rename encryption-at-rest related objects (#11059)

EncryptionConfig was renamed to EncryptedConfiguration and added to
the `apiserver.config.k8s.io` API group in Kubernetes 1.13.

The feature was previously in alpha and was not handling versions
properly, which lead to an originally unnoticed `v1` in the docs.

Also, the `--experimental-encryption-provider-config` flag is now called
just `--encryption-provider-config`.

* Documenting FlexVolume Resize alpha feature. (#10097)

* CR webhook conversion documentation (#10986)

* CR Conversion

* Addressing comments

* Addressing more comments

* Addressing even more comments

* Addressing even^2 more comments

* Remove references to etcd2 in v1.13 since support has been removed (#11414)

* Remove etcd2 references as etcd2 is deprecated

Link back to the v1.12 version of the etcd3 doc for
the etcd2->etcd3 migration instructions.

I updated the kube-apiserver reference manually,
unsure if that is auto-generated somehow.

The federation-apiserver can still potentially
support etcd2 so I didn't touch that.

* Remove outdated {master,node}.yaml files

There are master/node yaml files that reference
etcd2.service that are likely highly out of date.
I couldn't find any docs that actually reference
these templates so I removed them

* Address review comments

* Final Korean l10n work for dev-1.13 (#11440)

* Update outdated l10n(ko) contents (#11425)

fixes #11424

* Remove references to etcd2 in content/ko (#11416)

* Resolve conflicts against master for /ko contents (#11438)

* Fix unopened caution shortcode

* kubeadm: update the reference docs for 1.13 (#10960)

* docs update to promote TaintBasedEvictions to beta (#10765)

* First Korean l10n work for dev-1.13 (#10719)

* Update outdated l10n(ko) contents (#10689)

fixes #10686

* Translate concepts/overview/what-is-kubernetes in Korean (#10690)

* Translate concepts/overview/what-is-kubernetes in Korean

* Feedback from ClaudiaJKang

* Translate concepts/overview/components in Korean (#10882)

* Translate concepts/overview/components in Korean #10717

* Translate concepts/overview/components in Korean

* Translate concepts/overview/components in Korean

* Apply Korean glossary: 서비스 어카운트

* Translate concepts/overview/kubernetes-api in Korean (#10773)

* Translate concepts/overview/kubernetes-api in Korean

* Applied feedback from ianychoi

* kubeadm: update the configuration docs to v1beta1 (#10959)

* kubeadm: add small v1beta1 related updates (#10988)

* update new feature gates to document (#11295)

* Update dry run feature to beta (#11140)

* kubeadm: add improvements to HA docs (#11094)

* kubeadm: add information and diagrams for HA topologies

* kubeadm: update HA doc with simplified steps

* kubeadm: update HA doc with simplified steps

* edit ha, add new topology topic, reorder by weight

* troubleshoot markdown

* fix more markdown, fix links

* more markdown

* more markdown

* more markdown

* changes after reviewer comments

* add steps about Weave

* update note about stacked topology

* kubeadm: update reference docs

- add section about working with phases under kubeadm-init.md
- update GA / beta status of features
- kubeadm alpha phase was moved to kubeadm init phase
- new commands were added under kubeadm alpha
- included new CoreDNS usage examples

* Generate components and tools reference

* Add generated federation API Reference (#11491)

* Add generated federation API Reference

* Add front matter to federation reference

* Remove whitespace from federation front matter

* Remove more whitespace from federation front matter

* Remove superfluous kubefed reference

* Add frontmatter to generated kubefed reference

* Fix kubefed reference page frontmatter

* Generate kubectl reference docs 1.13 (#11487)

* Generate kubectl reference docs 1.13

* Fix links in kubectl reference

* Add 1.13 API reference (#11489)

* Update config.toml (#11486)

* Update config.toml

Preparing for 1.13 release, updating the config.toml and dropping the 1.8 docs reference.

* update dot releases and docsbranch typo

* adding .Site. to Params.currentUrl (#11503)

see #11502 for context

* Add 1.13 Release notes (#11499)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. language/en Issues or PRs related to English language lgtm "Looks good to me", indicates that a PR is ready to be merged. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.