New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for resizing PVs #284

Open
gnufied opened this Issue Apr 28, 2017 · 36 comments

Comments

Projects
@gnufied
Member

gnufied commented Apr 28, 2017

Feature Name

  • Add support for resizing existing PVs
  • Primary contact (assignee): Hemant Kumar( @gnufied)
  • Responsible SIGs: storage-sig
  • Design proposal link (community repo): kubernetes/community#657
  • Reviewer(s) - (for LGTM) recommend having 2+ reviewers: @saad-ali @jsafrane
  • Approver (likely from SIG/area to which feature belongs): @jsafrane
  • Feature target (which target equals to which milestone):
    • Alpha release target (x.y): 1.8
      • File system resizing and GCE, EBS, Cinder volume type support : 1.9
    • Beta release target (x.y): 1.11
    • Stable release target (x.y): 1.12
@duglin

This comment has been minimized.

duglin commented Apr 28, 2017

/cc @sandyamin123 @aniket-s-kulkarni

@idvoretskyi idvoretskyi added this to the v1.7 milestone May 3, 2017

@idvoretskyi

This comment has been minimized.

Member

idvoretskyi commented May 3, 2017

@gnufied can you update the feature description with the design proposal?

@idvoretskyi idvoretskyi added this to In Progress in Kubernetes 1.7 features May 3, 2017

@gnufied

This comment has been minimized.

Member

gnufied commented May 23, 2017

Updated link to design proposal - kubernetes/community#657

@saad-ali

This comment has been minimized.

Member

saad-ali commented May 25, 2017

This will not ship alpha until 1.8 at the earliest, moving out of 1.7 milestone

@saad-ali saad-ali modified the milestones: next-milestone, v1.7 May 25, 2017

@gnufied

This comment has been minimized.

Member

gnufied commented May 25, 2017

@saad-ali I know that timeline is pretty tight and code freeze is looming. But I would like to do my best to make this an alpha feature for 1.7. Yes - there is a chance, we may fail, but I don't see any harm in trying.

So - please, can we move this back to 1.7 milestone? I was hoping to have a discussion about this today in sig but we ran out of time. I am planning to schedule a call tomorrow morning.

@saad-ali

This comment has been minimized.

Member

saad-ali commented Jun 14, 2017

@saad-ali I know that timeline is pretty tight and code freeze is looming. But I would like to do my best to make this an alpha feature for 1.7. Yes - there is a chance, we may fail, but I don't see any harm in trying.

So - please, can we move this back to 1.7 milestone? I was hoping to have a discussion about this today in sig but we ran out of time. I am planning to schedule a call tomorrow morning.

I didn't mean to punt resize without your input @gnufied. Sorry about that. It just looked like there were a lot of big design questions, and I'd rather the team spend the time before 1.8 coding beings to close on those rather than rush the design and implementation. We can prioritize this higher for 1.8. Thanks for driving this!

@idvoretskyi

This comment has been minimized.

Member

idvoretskyi commented Jun 14, 2017

@gnufied I'd second @saad-ali's suggestion about not moving this item to 1.7 and keeping it for 1.8. Unfortunately, either feature or code freeze dates have already passed, so it is unlikely to add the new features now.

Would be happy to see this item in 1.8.

@gnufied

This comment has been minimized.

Member

gnufied commented Jun 14, 2017

@saad-ali @idvoretskyi no worries. Yeah perhaps it was bit over ambitious to try and get this at such a later stage. Looking forward to get this done for 1.8 though. 🍰 🍰

@saad-ali saad-ali modified the milestones: 1.8, next-milestone Jul 12, 2017

k8s-merge-robot added a commit to kubernetes/kubernetes that referenced this issue Sep 4, 2017

Merge pull request #49727 from gnufied/expand-pvc-plugin-changes
Automatic merge from submit-queue (batch tested with PRs 49727, 51792)

Implement Controller for growing persistent volumes

This PR implements API and controller plane changes necessary for doing controller side resize.

xref :  kubernetes/community#657 

Also xref kubernetes/enhancements#284 

```
Add alpha support for allowing users to grow persistent volumes. Currently we only support volume types that just require control plane resize (such as glusterfs) and don't need separate file system resize. 
```
@jdumars

This comment has been minimized.

Member

jdumars commented Sep 15, 2017

@gnufied do you have an update on missing docs? PR is due today.

@gnufied

This comment has been minimized.

Member

gnufied commented Sep 15, 2017

I will open docs PR EOD today. sorry about the delay.

@jdumars

This comment has been minimized.

Member

jdumars commented Sep 18, 2017

@gnufied any update on the docs PR?

@gnufied

This comment has been minimized.

Member

gnufied commented Sep 18, 2017

I am on it. It is taking longer than I expected, I am really sorry about that. But it should all be in by today.

@gnufied

This comment has been minimized.

Member

gnufied commented Oct 10, 2017

We partially implemented this in 1.8. we need to roll this feature over to 1.8. The feature design has already been merged.

@klausenbusk

This comment has been minimized.

klausenbusk commented Dec 27, 2017

Will this work/is there any plans to make this work with external provisioner (like the https://github.com/kubernetes-incubator/external-storage provisioners)?

@krmayankk

This comment has been minimized.

krmayankk commented Jan 9, 2018

is pvc resizing feature already supported for rbd(ceph) as well and did it already land in 1.9 ? @rootfs

@gnufied

This comment has been minimized.

Member

gnufied commented Jan 9, 2018

@krmayankk yes it works for rbd.

@gnufied

This comment has been minimized.

Member

gnufied commented Jan 9, 2018

@klausenbusk there is a plan to add resizing for flex volumes - #304 but there is no immediate plan to add resizing support in external provisioners.

k8s-merge-robot added a commit to kubernetes/kubernetes that referenced this issue Feb 5, 2018

Merge pull request #57017 from andyzhangx/azurefile-growsize
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

add PV size grow feature for azure file

**What this PR does / why we need it**:
According to kubernetes/enhancements#284, add size grow feature for azure file

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #56462 

**Special notes for your reviewer**:
Since azure file is using SMB 3.0 protocal, there is no necessary to resize filesystem on agent side, the agent node will detect the changed size automatically.

**Release note**:

```
add size grow feature for azure file
```
/sig azure
@gnufied @rootfs @brendandburns
@bradenwright

This comment has been minimized.

bradenwright commented Mar 14, 2018

So unless I'm missing something looks like Expanding Persistent Volumes Claims for gcePersistentDisk
& awsElasticBlockStore are both supported in 1.8 alpha

https://kubernetes.io/docs/concepts/storage/persistent-volumes/#aws

calebamiles pushed a commit to kubernetes/cloud-provider-azure that referenced this issue Mar 21, 2018

Merge pull request #57017 from andyzhangx/azurefile-growsize
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

add PV size grow feature for azure file

**What this PR does / why we need it**:
According to kubernetes/enhancements#284, add size grow feature for azure file

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #56462 

**Special notes for your reviewer**:
Since azure file is using SMB 3.0 protocal, there is no necessary to resize filesystem on agent side, the agent node will detect the changed size automatically.

**Release note**:

```
add size grow feature for azure file
```
/sig azure
@gnufied @rootfs @brendandburns

calebamiles pushed a commit to kubernetes/cloud-provider-gcp that referenced this issue Mar 21, 2018

Merge pull request #55815 from gnufied/implement-node-fs-resize
Automatic merge from submit-queue (batch tested with PRs 55545, 55548, 55815, 56136, 56185). If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Implement node fs resize

Implement kubelet side resizing of file system.

xref - kubernetes/enhancements#284 

```release-note
Implement kubelet side file system resizing. Also implement GCE PD resizing
```
@gnufied

This comment has been minimized.

Member

gnufied commented Apr 9, 2018

@bradenwright for GCE and AWS the resizing support was introduced in 1.9 alpha. 1.8 only had controller changes and as a result file system could not be resized. In 1.9 we introduced file system resizing and added support for gce/AWS etc. Obviously this is much more polished in 1.10

@childsb @saad-ali can we add this 1.11 milestone? This feature will go beta in 1.11

@saad-ali saad-ali modified the milestones: v1.9, v1.11 Apr 12, 2018

@justaugustus

This comment has been minimized.

Member

justaugustus commented Apr 17, 2018

@gnufied
Any plans for this in 1.11?

If so, can you please ensure the feature is up-to-date with the appropriate:

  • Description
  • Milestone
  • Assignee(s)
  • Labels:
    • stage/{alpha,beta,stable}
    • sig/*
    • kind/feature

cc @idvoretskyi

@gnufied

This comment has been minimized.

Member

gnufied commented Apr 24, 2018

@justaugustus as indicated above, we are going to take this feature to beta in 1.11 . I have already updated milestones for that

k8s-merge-robot added a commit to kubernetes/kubernetes that referenced this issue May 26, 2018

Merge pull request #64288 from gnufied/take-volume-resize-beta
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Move volume resize feature to beta

Move volume resizing feature to beta. 

xref kubernetes/enhancements#284

```release-note
Move Volume expansion to Beta
```
@justaugustus

This comment has been minimized.

Member

justaugustus commented Jun 4, 2018

@gnufied --
We're doing one more sweep of the 1.11 Features tracking spreadsheet.
Would you mind filling in any incomplete / blank fields for this feature's line item?

@gnufied

This comment has been minimized.

Member

gnufied commented Jun 4, 2018

yes We need new documentation for this feature - I have opened kubernetes/website#8896

k8s-merge-robot added a commit to kubernetes/kubernetes that referenced this issue Jun 5, 2018

Merge pull request #64386 from andyzhangx/azuredisk-sizegrow
Automatic merge from submit-queue. If you want to cherry-pick this change to another branch, please follow the instructions <a href="https://github.com/kubernetes/community/blob/master/contributors/devel/cherry-picks.md">here</a>.

Add azuredisk PV size grow feature

**What this PR does / why we need it**:
According to kubernetes/enhancements#284, add size grow feature for azure disk

**Which issue(s) this PR fixes** *(optional, in `fixes #<issue number>(, fixes #<issue_number>, ...)` format, will close the issue(s) when PR gets merged)*:
Fixes #56463

**Special notes for your reviewer**:
 - This feature is ony for azure managed disk, and if that disk is already attached to a running VM, disk resize will fail as following:
```
$ kubectl describe pvc pvc-azuredisk
Events:
  Type     Reason              Age               From           Message
  ----     ------              ----              ----           -------
  Warning  VolumeResizeFailed  51s (x3 over 3m)  volume_expand  Error expanding volume "default/pvc-azuredisk" of plugin kubernetes.io/azure-disk : disk.DisksClient#CreateOrUpdate: Failure responding to request: StatusCode=409 -- Original Error: autorest/azure: Service returned an error. Status=409 Code="OperationNotAllowed" Message="Cannot resize disk andy-mg1102-dynamic-pvc-d2d00dd9-6185-11e8-a6c3-000d3a0643a8 while it is attached to running VM /subscriptions/.../resourceGroups/.../providers/Microsoft.Compute/virtualMachines/k8s-agentpool-17607330-0."
```

**How to use this feature**
 - `kubectl edit pvc pvc-azuredisk` to change azuredisk PVC size from 6GB to 10GB
```
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  annotations:
...
    volume.beta.kubernetes.io/storage-provisioner: kubernetes.io/azure-disk
  creationTimestamp: 2018-05-27T08:13:23Z
  finalizers:
  - kubernetes.io/pvc-protection
  name: pvc-azuredisk
...
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 6Gi
  storageClassName: hdd
  volumeMode: Filesystem
  volumeName: pvc-d2d00dd9-6185-11e8-a6c3-000d3a0643a8
status:
  accessModes:
  - ReadWriteOnce
  capacity:
    storage: 6Gi
  conditions:
  - lastProbeTime: null
    lastTransitionTime: 2018-05-27T08:14:34Z
    message: Waiting for user to (re-)start a pod to finish file system resize of
      volume on node.
    status: "True"
    type: FileSystemResizePending
  phase: Bound
```

 - After resized, `/mnt/disk` is still 6GB
```
$ kubectl exec -it nginx-azuredisk -- bash
# df -h
Filesystem      Size  Used Avail Use% Mounted on
...
/dev/sdf        5.8G   15M  5.5G   1% /mnt/disk
...
```

 - After user run `sudo resize2fs /dev/sdf` in agent node, `/mnt/disk` becomes 10GB now:
```
$ kubectl exec -it nginx-azuredisk -- bash
# df -h
Filesystem      Size  Used Avail Use% Mounted on
...
/dev/sdf        9.8G   16M  9.3G   1% /mnt/disk
...
```

**Release note**:

```
Add azuredisk size grow feature
```

/sig azure
/assign @feiskyer @karataliu @gnufied 
cc @khenidak
@kacole2

This comment has been minimized.

Contributor

kacole2 commented Jul 23, 2018

@gnufied This feature was worked on in the previous milestone, so we'd like to check in and see if there are any plans for this to graduate stages in Kubernetes 1.12 as mentioned in the original post. This still has the 1.11 milestone as well so we need to update it accordingly.

If there are any updates, please explicitly ping @justaugustus, @kacole2, @robertsandoval, @rajendar38 to note that it is ready to be included in the Features Tracking Spreadsheet for Kubernetes 1.12.


Please note that the Features Freeze is July 31st, after which any incomplete Feature issues will require an Exception request to be accepted into the milestone.

In addition, please be aware of the following relevant deadlines:

  • Docs deadline (open placeholder PRs): 8/21
  • Test case freeze: 8/28

Please make sure all PRs for features have relevant release notes included as well.

Happy shipping!

@justaugustus justaugustus removed this from the v1.11 milestone Jul 31, 2018

@kacole2

This comment has been minimized.

Contributor

kacole2 commented Oct 8, 2018

Hi
This enhancement has been tracked before, so we'd like to check in and see if there are any plans for this to graduate stages in Kubernetes 1.13. This release is targeted to be more ‘stable’ and will have an aggressive timeline. Please only include this enhancement if there is a high level of confidence it will meet the following deadlines:

  • Docs (open placeholder PRs): 11/8
  • Code Slush: 11/9
  • Code Freeze Begins: 11/15
  • Docs Complete and Reviewed: 11/27

Please take a moment to update the milestones on your original post for future tracking and ping @kacole2 if it needs to be included in the 1.13 Enhancements Tracking Sheet

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment