[cinder-csi-plugin] Resize filesystem of a larger volume created from a snapshot/volume #1563

alibo · 2021-06-04T23:00:11Z

What this PR does / why we need it:

If CO requests a volume to be created from existing snapshot or volume and the requested size of the volume is larger than the original snapshotted (or cloned volume), the Plugin can either refuse such a call with OUT_OF_RANGE error or MUST provide a volume that, when presented to a workload by NodePublish call, has both the requested (larger) size and contains data from the snapshot (or original volume). Explicitly, it's the responsibility of the Plugin to resize the filesystem of the newly created volume at (or before) the NodePublish call, if the volume has VolumeCapability access type MountVolume and the filesystem resize is required in order to provision the requested capacity.
container-storage-interface/spec#452

Which issue this PR fixes(if applicable):
fixes #1539

Special notes for reviewers:

NeedResize() is added to k8s.io/mount-utils@v0.21.1, so I had to update it.

Release note:

[cinder-csi-plugin] Fixed the issue when expanding volume that is created from a snapshot or another volume.

k8s-ci-robot · 2021-06-04T23:00:14Z

Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please follow instructions at https://git.k8s.io/community/CLA.md#the-contributor-license-agreement to sign the CLA.

It may take a couple minutes for the CLA signature to be fully registered; after that, please reply here with a new comment and we'll verify. Thanks.

If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
If you signed the CLA as a corporation, please sign in with your organization's credentials at https://identity.linuxfoundation.org/projects/cncf to be authorized.
If you have done the above and are still having issues with the CLA being reported as unsigned, please log a ticket with the Linux Foundation Helpdesk: https://support.linuxfoundation.org/
Should you encounter any issues with the Linux Foundation Helpdesk, send a message to the backup e-mail support address at: login-issues@jira.linuxfoundation.org

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

k8s-ci-robot · 2021-06-04T23:00:19Z

Welcome @alibo!

It looks like this is your first PR to kubernetes/cloud-provider-openstack 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/cloud-provider-openstack has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

k8s-ci-robot · 2021-06-04T23:00:19Z

Hi @alibo. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

theopenlab-ci · 2021-06-04T23:04:28Z

Build succeeded.

cloud-provider-openstack-format : SUCCESS in 2m 35s

theopenlab-ci · 2021-06-04T23:04:40Z

Build succeeded.

cloud-provider-openstack-sanity-test-csi-cinder : SUCCESS in 2m 46s

theopenlab-ci · 2021-06-04T23:05:27Z

Build succeeded.

cloud-provider-openstack-unittest : SUCCESS in 3m 19s

alibo · 2021-06-04T23:12:44Z

I signed it

theopenlab-ci · 2021-06-04T23:17:35Z

Build succeeded.

cloud-provider-openstack-acceptance-test-keystone-authentication-authorization : SUCCESS in 15m 26s

theopenlab-ci · 2021-06-04T23:17:39Z

Build failed.

cloud-provider-openstack-acceptance-test-e2e-conformance : POST_FAILURE in 15m 06s

theopenlab-ci · 2021-06-04T23:22:03Z

Build failed.

cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.20 : POST_FAILURE in 15m 42s

theopenlab-ci · 2021-06-04T23:27:22Z

Build failed.

cloud-provider-openstack-multinode-csi-migration-test : FAILURE in 24m 50s

alibo · 2021-06-04T23:31:40Z

Worth mentioning I tried writing a new test in nodeserver_test.go, but I faced issues with existing mocked methods/structs:

OpenStackMock.GetVolume always returns

volumes.Volume{
	ID:               "261a8b81-3660-43e5-bab8-6470b65ee4e9",
	Name:             "fake-duplicate",
	Status:           "available",
	AvailabilityZone: "nova",
	Size:             1,
}

I've tried to fix it, but many tests failed

GetVolume in TestCreateVolumeFromSourceVolume should be mocked explicitly to avoid returning default data defined in OpenStackMock
Sometimes, the returned object is cached from previous tests because mocked objects are defined in init().
ResizeFS in mount-utils is better to be mocked and injected into nodeserver.go, so its method calls can be easily tested in NodeStageVolume and NodeExpandVolume methods by mocking them instead of using FakeExec commands (which is also used in mocked version of Mounter() method) which (IMHO) would test the functionality of resize2fs pkg itself.

theopenlab-ci · 2021-06-04T23:34:29Z

Build failed.

cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.19 : POST_FAILURE in 15m 42s

alibo · 2021-06-04T23:45:03Z

Build failed.

* [cloud-provider-openstack-multinode-csi-migration-test ](https://logs.openlabtesting.org/logs/63/1563/0fcf03706514d1c3d2261259915b60746f5c180f/cloud-provider-openstack-multinode-csi-migration-test/cloud-provider-openstack-multinode-csi-migration-test/f1a9c8a/) : FAILURE in 24m 50s

  Warning  Failed     3m55s (x3 over 4m46s)  kubelet            Failed to pull image "192.168.0.6:32000/k8scloudprovider/cinder-csi-plugin:latest": rpc error: code = Unknown desc = Error response from daemon: Get http://192.168.0.6:32000/v2/: dial tcp 192.168.0.6:32000: connect: no route to host
  Warning  Failed     3m55s (x3 over 4m46s)  kubelet            Error: ErrImagePull
  Normal   BackOff    3m28s (x4 over 4m46s)  kubelet            Back-off pulling image "192.168.0.6:32000/k8scloudprovider/cinder-csi-plugin:latest"
  Warning  Failed     3m28s (x4 over 4m46s)  kubelet            Error: ImagePullBackOff

it's weired 🤔

alibo · 2021-06-04T23:48:58Z

/test cloud-provider-openstack-multinode-csi-migration-test

k8s-ci-robot · 2021-06-04T23:49:11Z

@alibo: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/test cloud-provider-openstack-multinode-csi-migration-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

theopenlab-ci · 2021-06-05T00:12:58Z

Build failed.

cloud-provider-openstack-acceptance-test-csi-manila : POST_FAILURE in 1h 09m 38s

theopenlab-ci · 2021-06-05T02:03:49Z

Build failed.

cloud-provider-openstack-e2e-test-csi-cinder : TIMED_OUT in 3h 01m 31s

theopenlab-ci · 2021-06-05T02:51:35Z

Build failed.

cloud-provider-openstack-multinode-csi-migration-test : TIMED_OUT in 3h 01m 41s

jichenjc · 2021-06-07T00:27:20Z

/ok-to-test

theopenlab-ci · 2021-06-18T11:24:38Z

Build succeeded.

cloud-provider-openstack-acceptance-test-keystone-authentication-authorization : SUCCESS in 32m 00s

ramineni · 2021-06-18T12:46:22Z

/test cloud-provider-openstack-multinode-csi-migration-test

k8s-ci-robot · 2021-06-18T12:46:23Z

@ramineni: The specified target(s) for /test were not found.
The following commands are available to trigger jobs:

/test pull-cloud-provider-openstack-check
/test pull-cloud-provider-openstack-test

Use /test all to run all jobs.

In response to this:

/test cloud-provider-openstack-multinode-csi-migration-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

ramineni · 2021-06-18T12:46:42Z

/test cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.19

k8s-ci-robot · 2021-06-18T12:46:43Z

@ramineni: The specified target(s) for /test were not found.
The following commands are available to trigger jobs:

/test pull-cloud-provider-openstack-check
/test pull-cloud-provider-openstack-test

Use /test all to run all jobs.

In response to this:

/test cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.19

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

theopenlab-ci · 2021-06-18T12:53:49Z

Build failed.

cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.19 : POST_FAILURE in 6m 06s

theopenlab-ci · 2021-06-18T13:13:41Z

Build succeeded.

cloud-provider-openstack-test-occm-e2e-conformance : SUCCESS in 2h 21m 20s

theopenlab-ci · 2021-06-18T13:30:12Z

Build succeeded.

cloud-provider-openstack-multinode-csi-migration-test : SUCCESS in 42m 52s

alibo · 2021-06-19T06:38:40Z

/test cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.19 🙏📿🛐

k8s-ci-robot · 2021-06-19T06:38:54Z

@alibo: The specified target(s) for /test were not found.
The following commands are available to trigger jobs:

/test pull-cloud-provider-openstack-check
/test pull-cloud-provider-openstack-test

Use /test all to run all jobs.

In response to this:

/test cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.19 🙏📿🛐

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

alibo · 2021-06-19T07:09:05Z

@ramineni do you have any idea how I can fix cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.19 test? It's the only one is failed and I guess it's not related to this change since it's for another branch.

alibo · 2021-06-19T08:41:28Z

@jichenjc @lingxiankong can you please review it again?

lingxiankong · 2021-06-19T09:50:49Z

/override openlab/cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.19

@alibo Don't worry about that job, it needs to be fixed.

k8s-ci-robot · 2021-06-19T09:50:52Z

@lingxiankong: Overrode contexts on behalf of lingxiankong: openlab/cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.19

In response to this:

/override openlab/cloud-provider-openstack-acceptance-test-e2e-conformance-stable-branch-v1.19

@alibo Don't worry about that job, it needs to be fixed.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

lingxiankong · 2021-06-19T09:52:51Z

/lgtm

ramineni · 2021-06-21T05:01:06Z

@alibo Thanks for the PR.

/lgtm
/approve

k8s-ci-robot · 2021-06-21T05:01:15Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ramineni

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [ramineni]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

lingxiankong · 2021-06-22T04:41:41Z

@ramineni Is this one a good candidate to backport?

ramineni · 2021-06-23T10:17:14Z

@lingxiankong I'm not sure the updated package 1.21.1 will be compatible with 1.20 kubernetes and other packages. Lets hold it until someone needs it.

lingxiankong · 2021-06-23T11:09:45Z

@lingxiankong I'm not sure the updated package 1.21.1 will be compatible with 1.20 kubernetes and other packages. Lets hold it until someone needs it.

ack

…ubernetes#1563)

k8s-ci-robot added the release-note-none Denotes a PR that doesn't merit a release note. label Jun 4, 2021

k8s-ci-robot added the cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. label Jun 4, 2021

k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Jun 4, 2021

k8s-ci-robot requested review from adisky and ramineni June 4, 2021 23:00

k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jun 4, 2021

alibo mentioned this pull request Jun 4, 2021

[cinder-csi-plugin] filesystem of a larger volume created from a snapshot/volume is not expanded #1539

Closed

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels Jun 4, 2021

k8s-ci-robot added the ok-to-test Indicates a non-member PR verified by an org member that is safe to test. label Jun 7, 2021

k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed release-note-none Denotes a PR that doesn't merit a release note. labels Jun 19, 2021

k8s-ci-robot assigned lingxiankong Jun 19, 2021

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 19, 2021

k8s-ci-robot assigned ramineni Jun 21, 2021

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 21, 2021

k8s-ci-robot merged commit 6c34dc4 into kubernetes:master Jun 21, 2021

alibo deleted the fix/snapshot-expansion branch June 21, 2021 05:36

powellchristoph pushed a commit to powellchristoph/cloud-provider-openstack that referenced this pull request Jan 19, 2022

Resize filesystem of a larger volume created from a snapshot/volume (k…

a7744fb

…ubernetes#1563)

RomanBednar mentioned this pull request Feb 8, 2023

Filesystem resize is required after snapshot restore / volume clone kubernetes/kubernetes#94929

Closed

6 tasks

[cinder-csi-plugin] Resize filesystem of a larger volume created from a snapshot/volume #1563

[cinder-csi-plugin] Resize filesystem of a larger volume created from a snapshot/volume #1563

Conversation

alibo commented Jun 4, 2021 • edited by lingxiankong

k8s-ci-robot commented Jun 4, 2021

k8s-ci-robot commented Jun 4, 2021

k8s-ci-robot commented Jun 4, 2021

theopenlab-ci bot commented Jun 4, 2021

theopenlab-ci bot commented Jun 4, 2021

theopenlab-ci bot commented Jun 4, 2021

alibo commented Jun 4, 2021

theopenlab-ci bot commented Jun 4, 2021

theopenlab-ci bot commented Jun 4, 2021

theopenlab-ci bot commented Jun 4, 2021

theopenlab-ci bot commented Jun 4, 2021

alibo commented Jun 4, 2021

theopenlab-ci bot commented Jun 4, 2021

alibo commented Jun 4, 2021

alibo commented Jun 4, 2021

k8s-ci-robot commented Jun 4, 2021

theopenlab-ci bot commented Jun 5, 2021

theopenlab-ci bot commented Jun 5, 2021

theopenlab-ci bot commented Jun 5, 2021

jichenjc commented Jun 7, 2021

theopenlab-ci bot commented Jun 18, 2021

ramineni commented Jun 18, 2021

k8s-ci-robot commented Jun 18, 2021

ramineni commented Jun 18, 2021

k8s-ci-robot commented Jun 18, 2021

theopenlab-ci bot commented Jun 18, 2021

theopenlab-ci bot commented Jun 18, 2021

theopenlab-ci bot commented Jun 18, 2021

alibo commented Jun 19, 2021

k8s-ci-robot commented Jun 19, 2021

alibo commented Jun 19, 2021

alibo commented Jun 19, 2021

lingxiankong commented Jun 19, 2021

k8s-ci-robot commented Jun 19, 2021

lingxiankong commented Jun 19, 2021

ramineni commented Jun 21, 2021

k8s-ci-robot commented Jun 21, 2021

lingxiankong commented Jun 22, 2021

ramineni commented Jun 23, 2021

lingxiankong commented Jun 23, 2021

alibo commented Jun 4, 2021 •

edited by lingxiankong