Deploy Cinder CSI driver to provision volumes over OpenStack #5184

alijahnas · 2019-09-18T07:29:54Z

What type of PR is this?
/kind feature

What this PR does / why we need it:
Deploys Cinder CSI driver in order to provision volumes from OpenStack Cinder.

Which issue(s) this PR fixes:
Fixes #5183

Special notes for your reviewer:
I reused parts of the code from OpenStack internal provisioner and renamed variables to from openstack_ to cinder_ in order to have backward compatibility and the possibility to use both in-tree OpenStack provisioner and Cinder CSI driver.
I used the info from https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/using-cinder-csi-plugin.md to implement and test the driver.

roles/kubernetes-apps/persistent_volumes/cinder is coming in another PR.
Documentation is coming if needed.

Does this PR introduce a user-facing change?:
None

alijahnas · 2019-09-23T08:45:20Z

@holmsten @mirwan
Hi! The failure in Kubespray CI - tf-ovh_ubuntu18-calico doesn't seem to be linked to what I propose, am I wrong? Thanks for the review!

Miouge1 · 2019-09-25T12:35:45Z

Thank you for the contribution. I'll retrigger that CI job, the error might be related to an outside cause.

Regarding the content of the PR, is that the external openstack provider?

alijahnas · 2019-09-25T13:17:50Z

Hi @Miouge1 thanks for the reply 👍

It's kind of the new external openstack provider for volumes.

Cloud controllers (like the OpenStack one) should run outside Kubernetes now, but even though they are, the volume provisioning isn't done by the external cloud controller, as specified here: https://kubernetes.io/docs/tasks/administer-cluster/running-cloud-controller/#support-for-volumes

The right way to implement cloud volume provisioning in Kubernetes now is through CSI drivers: https://kubernetes-csi.github.io/docs/introduction.html

The OpenStack Cinder CSI driver is part of the openstack external cloud controller project: https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/using-cinder-csi-plugin.md

I can also implement the OpenStack external cloud controller in Kubespray if you would like, it shouldn't be much effort. But it would only be used for Load Balancing. The Volume provisioning is in the CSI driver now.

Ask me if you want more details about it.
Thanks

Miouge1 · 2019-09-25T13:29:32Z

Great! I was looking into https://github.com/kubernetes/cloud-provider-openstack last week.

I'm definitely interested in getting this in and deprecating the in-tree cloud provider.

Will there be a migration path from the in-tree provider to the out of tree provider?

alijahnas · 2019-09-25T13:57:59Z

Using CSI driver doesn't conflict with the old in-tree cloud provider as the provisioner has another name. By the way the code for the csi provisioner is ready on my kubespray fork. I am just waiting for this PR to be in.

For the in-tree to out-tree cloud provider OpenStack, the configuration during KubeADM should be changed so that the right flags are set on kube-controller and kube-apiserver. These flags will also be deprecated and removed from kubernetes once everybody is sure that they have all their calls out of tree.

alijahnas · 2019-10-08T14:19:54Z

@Miouge1 @mirwan @holmsten
Hi ! Any updates on the review of this PR please ? Thanks

Miouge1

I have tested this and it works well. To be noted that the kind: StorageClass needs to be created with provisioner: cinder.csi.openstack.org.

I suggest to do something like in roles/kubernetes-apps/persistent_volumes/openstack/templates/openstack-storage-class.yml.j2 to make it easier to consumers.

Also I wonder how people who are currently using the in-tree provider will be able to transition. Can we add a note somewhere in a .md file for that?

Miouge1 · 2019-10-11T10:25:03Z

roles/kubernetes-apps/csi_driver/cinder/templates/cinder-csi-controllerplugin.yml.j2

+  name: csi-cinder-controllerplugin
+  namespace: kube-system
+spec:
+  replicas: 1


Can we make replicas configurable with a group_var?

inventory/sample/group_vars/all/all.yml

alijahnas · 2019-10-11T11:09:36Z

I have tested this and it works well. To be noted that the kind: StorageClass needs to be created with provisioner: cinder.csi.openstack.org.

I suggest to do something like in roles/kubernetes-apps/persistent_volumes/openstack/templates/openstack-storage-class.yml.j2 to make it easier to consumers.

Also I wonder how people who are currently using the in-tree provider will be able to transition. Can we add a note somewhere in a .md file for that?

Hi,
I have a branch in my fork that implements the provisioner as you suggest: https://github.com/alijahnas/kubespray/tree/feature/cinder-csi-provisioner
I was just waiting for this PR to merge so that I can rebase and make a new PR for the provisioner to separate things.

I can write some doc to ease the usage and transition.

Miouge1 · 2019-10-11T11:34:36Z

Let's try to get some basic doc and the StorageClass thing in this PR if you don't mind.

alijahnas · 2019-10-11T12:09:23Z

@Miouge1 Ok no worries, I'll try to do it as fast as I can cause I will be out of office for a week from tomorrow with limited access to electronics, other than what is found on a sailing ship :D

alijahnas · 2019-10-11T15:33:00Z

@Miouge1
I added the provisioner and written the documentation.

alijahnas · 2019-10-22T09:51:36Z

@Miouge1
Hi!
I don't think the errors are related to the code I uploaded.

Miouge1 · 2019-10-22T14:19:37Z

@alijahnas OK I'm re-running that job 🤞

alijahnas · 2019-10-30T09:35:11Z

Hi @Miouge1
The CI platform doesn't want to show me love with beautiful green checks. The red crosses are not my fault >_<

holmsten · 2019-10-30T13:20:38Z

@alijahnas can you rebase from master?

alijahnas · 2019-10-30T13:56:36Z

@alijahnas can you rebase from master?

Done, thanks!

holmsten · 2019-10-30T14:52:06Z

/lgtm

riverzhang · 2019-11-01T07:59:03Z

/approve

k8s-ci-robot · 2019-11-01T07:59:14Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: alijahnas, riverzhang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [riverzhang]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

* 'master' of https://github.com/kubernetes-sigs/kubespray: (28 commits) Do not run kubeadm upgrade on first deploy (kubernetes-sigs#5339) Added Amazon Linux 2 support for deploying with docker (kubernetes-sigs#5301) Set certs and files with kubeadm token to mode 0640 (kubernetes-sigs#5325) Fix calico name resolution (kubernetes-sigs#5291) library files added to setup.cfg (kubernetes-sigs#5274) Fix helm for Kubernetes 1.16.2 (kubernetes-sigs#5332) Pass `ingress_nginx_extra_args` when deploying the nginx-ingress addon (kubernetes-sigs#5321) Respect cri-tool supported version matrix (kubernetes-sigs#5241) fix useradd etcd (kubernetes-sigs#5281) Fix indentation in cilium-ds.yml template (kubernetes-sigs#5305) Remove default docker_options from sample (kubernetes-sigs#5287) Add cilium_tunnel_mode variable to the cilium config (kubernetes-sigs#5295) Fixes issue kubernetes-sigs#5299 (kubernetes-sigs#5300) containerd installation on Debian (kubernetes-sigs#5326) Skip coredns image when evaluating kubeadm images (kubernetes-sigs#5327) [FIX] fix incorrect link to downloads documentation (kubernetes-sigs#5319) Remove unnecessary and risky reload network for resolvconf propagation (kubernetes-sigs#5322) Enable systemd-resolved DNS resolution mode (kubernetes-sigs#5318) Scale down coredns created by kubeadm upgrade to 0 replicas (kubernetes-sigs#5308) Deploy Cinder CSI driver to provision volumes over OpenStack (kubernetes-sigs#5184) ...

…tes-sigs#5184) * Deploy Cinder CSI driver to provision volumes over OpenStack * Deploy Cinder CSI StorageClass * Cinder CSI doc

k8s-ci-robot added kind/feature Categorizes issue or PR as related to a new feature. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Sep 18, 2019

k8s-ci-robot requested review from holmsten and mirwan September 18, 2019 07:30

alijahnas mentioned this pull request Sep 18, 2019

Using cinder as storage class with cloud-provider-openstack kubernetes/cloud-provider-openstack#758

Closed

k8s-ci-robot assigned alijahnas Sep 24, 2019

alijahnas removed their assignment Sep 24, 2019

Miouge1 suggested changes Oct 11, 2019

View reviewed changes

Miouge1 approved these changes Oct 22, 2019

View reviewed changes

Ali Sanhaji added 3 commits October 30, 2019 14:52

Deploy Cinder CSI driver to provision volumes over OpenStack

af2ac25

Deploy Cinder CSI StorageClass

da1e21d

Cinder CSI doc

f9bb377

alijahnas force-pushed the feature/cinder-csi-plugin branch from 1cb596c to f9bb377 Compare October 30, 2019 13:54

k8s-ci-robot assigned holmsten Oct 30, 2019

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 30, 2019

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 1, 2019

k8s-ci-robot merged commit b0ee1f6 into kubernetes-sigs:master Nov 1, 2019

alijahnas deleted the feature/cinder-csi-plugin branch November 3, 2019 22:02

Miouge1 mentioned this pull request Dec 5, 2019

Release Proposal v2.12.0 #5409

Closed

alijahnas mentioned this pull request Feb 18, 2020

REQUEST: New membership for alijahnas kubernetes/org#1635

Closed

6 tasks

This was referenced Mar 26, 2020

Azure Disk CSI deployment #5833

Merged

GCP Persistent Disk CSI Driver deployment #5857

Merged

pierreyves-lebrun mentioned this pull request Apr 1, 2020

Add vSphere external cloud provider #5861

Closed

huxcrux mentioned this pull request Apr 29, 2020

Release Proposal v2.13.0 #5962

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Deploy Cinder CSI driver to provision volumes over OpenStack #5184

Deploy Cinder CSI driver to provision volumes over OpenStack #5184

alijahnas commented Sep 18, 2019 •

edited

Loading

alijahnas commented Sep 23, 2019

Miouge1 commented Sep 25, 2019

alijahnas commented Sep 25, 2019

Miouge1 commented Sep 25, 2019

alijahnas commented Sep 25, 2019

alijahnas commented Oct 8, 2019

Miouge1 left a comment

Miouge1 Oct 11, 2019

alijahnas commented Oct 11, 2019

Miouge1 commented Oct 11, 2019

alijahnas commented Oct 11, 2019

alijahnas commented Oct 11, 2019 •

edited

Loading

alijahnas commented Oct 22, 2019

Miouge1 commented Oct 22, 2019

alijahnas commented Oct 30, 2019

holmsten commented Oct 30, 2019

alijahnas commented Oct 30, 2019

holmsten commented Oct 30, 2019

riverzhang commented Nov 1, 2019

k8s-ci-robot commented Nov 1, 2019

Deploy Cinder CSI driver to provision volumes over OpenStack #5184

Deploy Cinder CSI driver to provision volumes over OpenStack #5184

Conversation

alijahnas commented Sep 18, 2019 • edited Loading

alijahnas commented Sep 23, 2019

Miouge1 commented Sep 25, 2019

alijahnas commented Sep 25, 2019

Miouge1 commented Sep 25, 2019

alijahnas commented Sep 25, 2019

alijahnas commented Oct 8, 2019

Miouge1 left a comment

Choose a reason for hiding this comment

Miouge1 Oct 11, 2019

Choose a reason for hiding this comment

alijahnas commented Oct 11, 2019

Miouge1 commented Oct 11, 2019

alijahnas commented Oct 11, 2019

alijahnas commented Oct 11, 2019 • edited Loading

alijahnas commented Oct 22, 2019

Miouge1 commented Oct 22, 2019

alijahnas commented Oct 30, 2019

holmsten commented Oct 30, 2019

alijahnas commented Oct 30, 2019

holmsten commented Oct 30, 2019

riverzhang commented Nov 1, 2019

k8s-ci-robot commented Nov 1, 2019

alijahnas commented Sep 18, 2019 •

edited

Loading

alijahnas commented Oct 11, 2019 •

edited

Loading