csi-cinder-plugin ephemeral volume is not attached

[creatica-soft]

[cinder-csi-plugin] Volume status must be available to reserve, but the status is creating
/kind bug

What happened:
When using ephemeral volumes, the following error is logged:

Warning FailedMount 6s (x2 over 2m8s) kubelet, ds-kube-minion-2 (combined from similar events): MountVolume.SetUp failed for volume "tmp" : rpc error: code = Internal desc = Ephermal Volume Attach: Failed to Attach Volume: failed to attach 83a34327-0c7b-4b19-b770-e267e57b335b volume to 8e185cfe-a865-410d-bb1f-8888711de794 compute: Bad request with: [POST https://openstack.lat.corp-apps.com:8774/v2.1/servers/8e185cfe-a865-410d-bb1f-8888711de794/os-volume_attachments], error message: {"badRequest": {"message": "Invalid input received: Invalid volume: Volume status must be available to reserve, but the status is creating. (HTTP 400) (Request-ID: req-0b03bd41-116e-4f9c-9d84-0909b59dbe7a)", "code": 400}}

It seems that the plugin does not wait for the volume to become available, fails and goes to create it over and over again, so after few minutes I had about a dozen of available ephemeral volumes in openstack but none of them gets attached to the pod.

What you expected to happen:

Wait for the volume status to become available.

How to reproduce it:

create a pod similar to https://github.com/kubernetes/cloud-provider-openstack/blob/master/examples/cinder-csi-plugin/inline/inline-example.yaml

Anything else we need to know?:

Environment:

• openstack-cloud-controller-manager(or other related binary) version: latest
• OpenStack version: Pike
• Others: k8s 1.18

[ramineni]

@creatica-soft Thanks for reporting the issue. Ill check it

/assign

[jichenjc]

like a race issue , maybe we need follow https://github.com/kubernetes/cloud-provider-openstack/blob/master/pkg/csi/cinder/openstack/openstack_snapshots.go#L156
to add something for the volume as well...

[ramineni]

@creatica-soft Please check , it should be resolved by latest image

[creatica-soft]

@ramineni Thanks. It works if the volume gets attached but again when a volume fails to attach for any reason, the plugin requests a new volume and again in a few minutes, there will be few available volumes in openstack and non of them attached. I had to delete a deployment to stop this volumes acumulation in openstack and then delete those volumes manually from openstack. The current problem is that I can't attach more than 26 volumes from vda to vdz in centos VM. Could this ticket be reopen or should I open a new one?