Volume Plugin for Cinder; Openstack Block Storage #6689

spothanis · 2015-04-10T17:50:11Z

The PR implements cinder as a volume plugin. Cinder is openstack's block storage component.
The functionality is very similar to GCE's pd

lavalamp · 2015-04-10T17:58:10Z

To @thockin to find a reviewer.

markturansky · 2015-04-10T19:19:15Z

pkg/api/types.go

+// CinderPersistentDisk represents a Persistent Disk resource in Openstack.
+// A Cinder volume must exist and be formatted before mounting to a container.
+// The disk must also be in the same region as the kubelet.
+type CinderPersistentDiskVolumeSource struct {


Maybe just "CinderVolumeSource"? It will be present in both the VolumeSource and PersistentVolumeSource structs.

Will update

Yeah "PersistentDisk" is not a general concept name, it's literally the product name on Google Cloud Platform. If it's not part of the well-known name of this concept, don't include the words.

thockin · 2015-04-10T23:20:40Z

pkg/api/types.go

+// The disk must also be in the same region as the kubelet.
+type CinderPersistentDiskVolumeSource struct {
+	// Unique name of the PD resource. Used to identify the disk in cinder volume
+	PDName string `json:"pdName"`


As above - use field names that make sense to someone who knows this concept.

thockin · 2015-04-10T23:27:05Z

Thanks for taking point Mark. I had almost nothing to add.

markturansky · 2015-04-10T23:30:31Z

I'm happy to contribute where I can :)

thockin · 2015-04-15T17:23:37Z

@zmerlynn

This is another of the volume plugins for which we have no e2e, no way to add e2e in GCE, and no way to ingest 3rd party e2e. :(

pmorie · 2015-04-15T17:25:09Z

This is another of the volume plugins for which we have no e2e, no way to add e2e in GCE, and no way to ingest 3rd party e2e. :(

+1 (-1?)

wattsteve · 2015-05-07T02:00:30Z

When the Pods are scheduled on Kubernetes Nodes, what are the package requirements in order for the Kubernetes Nodes to be able mount the Cinder Block Device? For example, with the current Kubernetes GlusterFS and Ceph Plugins, glusterfs-client and ceph-common have to be installed, respectively.

spothanis · 2015-05-07T02:48:05Z

Here are some details about the requirements
https://github.com/spothanis/kubernetes/blob/cinder-vol-plugin/examples/mysql-cinder-pd/README.md

wattsteve · 2015-05-07T12:07:18Z

@spothanis thanks. The README is part of the PR so I had read it, but all it states is "Ensure cinder is installed and configured properly in the region in which kubelet is spun up". Does this plugin design assume that the kubelet is running in a Kubernetes deployment within an OpenStack cluster? Does it still work if Kubernetes is running adjacent to an OpenStack Cinder Deployment (a private cloud scenario)? What client libraries need to be installed on the Kubelet Hosts for the Volume Plugin to work?

spothanis · 2015-05-07T17:36:43Z

The volume plugin uses gopher cloud's openstack client. It is already
packaged into kubelet, so there are no additional installation requirements
for any client libraries.

Does this plugin design assume that the kubelet is running in a Kubernetes
deployment within an OpenStack cluster?
Yes.

Does it still work if Kubernetes is running adjacent to an OpenStack
Cinder Deployment (a private cloud scenario)?
Kubelet hosts must be nova vm's and must be in a region where Cinder is
available, only then nova will be able to attach a volume to the VM on
which kubelet has been running.

On Thu, May 7, 2015 at 5:07 AM, Steve Watt notifications@github.com wrote:

@spothanis https://github.com/spothanis thanks. The README is part of
the PR so I had read it, but all it states is "Ensure cinder is installed
and configured properly in the region in which kubelet is spun up". Does
this plugin design assume that the kubelet is running in a Kubernetes
deployment within an OpenStack cluster? Does it still work if Kubernetes is
running adjacent to an OpenStack Cinder Deployment (a private cloud
scenario)? What client libraries need to be installed on the Kubernetes for
the Volume Plugin to work?

—
Reply to this email directly or view it on GitHub
#6689 (comment)
.

Thank you,
Sreekanth

k8s-bot · 2015-05-22T00:04:18Z

Can one of the admins verify that this patch is reasonable to test? (reply "ok to test", or if you trust the user, reply "add to whitelist")

If this message is too spammy, please complain @ixdy.

erictune · 2015-06-01T14:58:04Z

ok to test

k8s-bot · 2015-06-01T14:59:11Z

EXPERIMENTAL JENKINS PR BUILDER: e2e build failed.

thockin · 2015-06-01T21:09:49Z

pkg/api/types.go

@@ -205,6 +205,8 @@ type VolumeSource struct {
 	Glusterfs *GlusterfsVolumeSource `json:"glusterfs"`
 	// PersistentVolumeClaimVolumeSource represents a reference to a PersistentVolumeClaim in the same namespace
 	PersistentVolumeClaimVolumeSource *PersistentVolumeClaimVolumeSource `json:"persistentVolumeClaim,omitempty"`
+	// CinderVolume represents a cinder volume attached and mounted on kubelets host machine
+	CinderVolume *CinderVolumeSource `json:"cinderVolume"`


do you really want "cinderVolume" to be the API or do you want "cinder" ?

I will change it to cinder

k8s-bot · 2015-07-08T22:55:09Z

Can one of the admins verify that this patch is reasonable to test? (reply "ok to test", or if you trust the user, reply "add to whitelist")

If this message is too spammy, please complain to ixdy.

pmorie · 2015-07-08T23:19:03Z

ok to test

On Wed, Jul 8, 2015 at 6:55 PM, Kubernetes Bot notifications@github.com
wrote:

Can one of the admins verify that this patch is reasonable to test? (reply
"ok to test", or if you trust the user, reply "add to whitelist")

If this message is too spammy, please complain to ixdy.

—
Reply to this email directly or view it on GitHub
#6689 (comment)
.

saad-ali · 2015-08-27T18:51:26Z

@spothanis If you plan to address @pmorie's feedback in a follow up PR, please open an issue and link it here for tracking. And thanks for all your hard work!

spothanis · 2015-08-27T19:23:00Z

@saad-ali I am rebasing to include @pmorie suggestions, will have it shortly, but if it is better to do it in a separate PR, I can do that too

pmorie · 2015-08-27T19:34:46Z

@spothanis It's your call -- I'll remove LGTM if you would prefer and add it again when you're ready, or I can review a follow up PR. For the record, I didn't mean to derail this so close to LGTM.

spothanis · 2015-08-27T19:40:54Z

@pmorie I really appreciate your time and advice on the PR. I will send a follow up PR to address the feedback
@thockin @saad-ali @markturansky Thank you for all of your time and feedback.

pmorie · 2015-08-27T21:08:52Z

@spothanis did you decide about a follow up or integrating comments now?

spothanis · 2015-08-27T21:23:27Z

@pmorie do a follow up PR if it is ok

pmorie · 2015-08-27T21:24:42Z

@spothanis sounds great.

@k8s-oncall Can oncall take a look at this PR?

k8s-github-robot · 2015-08-27T22:01:19Z

Labelling this PR as size/XXL

bprashanth · 2015-08-28T01:07:47Z

Fyi: plan is to wait for the submit queue to pick this up

pmorie · 2015-08-28T13:14:23Z

@bprashanth when do you expect that to happen?

bprashanth · 2015-08-28T16:41:02Z

Hopefully today, it's a function of how green the build is. Let me know if this is blocking you and we can do a manual merge, as it seems retty isolated.

saad-ali · 2015-08-28T17:46:01Z

@bprashanth Let's prioritize getting this in. Thanks!

Volume Plugin for Cinder; Openstack Block Storage

bprashanth · 2015-08-28T17:56:22Z

Grr, broke precommit, please /hack/verify-generated-conversions.sh again

spothanis · 2015-08-28T17:59:04Z

@bprashanth looking into it

liggitt · 2015-08-28T18:23:13Z

how did shippable pass with broken conversions?

bprashanth · 2015-08-28T18:24:52Z

the results should expire. it passed before the breaking changes came in.

pmorie · 2015-08-28T18:30:04Z

@bprashanth Did I read your comment correctly to mean that the precommit hook changed after the last shippable run?

bprashanth · 2015-08-28T18:49:02Z

the generated code changed beneath us in a way that didn't cause a merge conflict but caused the precommit to fail, but we didn't notice because shippable ran and passed before the change

spothanis · 2015-08-28T22:14:49Z

I will open a new PR with the required changes

wattsteve · 2015-08-31T00:47:22Z

@spothanis et al. Thanks for all your hard work on this. This is an awesome feature to add!

Volume Plugin for Cinder; Openstack Block Storage

googlebot added the cla: yes label Apr 10, 2015

lavalamp assigned thockin Apr 10, 2015

markturansky reviewed Apr 10, 2015
View reviewed changes

spothanis force-pushed the cinder-vol-plugin branch from 3e7052c to 4dc48ac Compare April 10, 2015 19:34

thockin reviewed Apr 10, 2015
View reviewed changes

spothanis force-pushed the cinder-vol-plugin branch from 4dc48ac to 694e08d Compare April 15, 2015 16:39

spothanis force-pushed the cinder-vol-plugin branch from 694e08d to e78d268 Compare April 21, 2015 17:20

erictune added this to the v1.0-post milestone Jun 1, 2015

thockin reviewed Jun 1, 2015
View reviewed changes

pmorie added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 27, 2015

k8s-github-robot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Aug 27, 2015

bprashanth added the ok-to-merge label Aug 27, 2015

jsafrane mentioned this pull request Aug 28, 2015

RFC: refactor persistent storage example and add Cinder documentation openshift/origin#4403

Merged

bprashanth added a commit that referenced this pull request Aug 28, 2015

Merge pull request #6689 from spothanis/cinder-vol-plugin

8d0d54f

Volume Plugin for Cinder; Openstack Block Storage

bprashanth merged commit 8d0d54f into kubernetes:master Aug 28, 2015

bprashanth mentioned this pull request Aug 28, 2015

Revert "Volume Plugin for Cinder; Openstack Block Storage" #13316

Merged

spothanis mentioned this pull request Aug 31, 2015

Cinder Volume Plugin #13367

Merged

BugRoger mentioned this pull request Sep 10, 2015

Pass cloudProvider from Kubelet to aws_ebs volume plugin #13784

Merged

dims pushed a commit to dims/kubernetes that referenced this pull request Feb 8, 2018

Merge pull request kubernetes#6689 from spothanis/cinder-vol-plugin

c35959c

Volume Plugin for Cinder; Openstack Block Storage

Volume Plugin for Cinder; Openstack Block Storage #6689

Volume Plugin for Cinder; Openstack Block Storage #6689

Conversation

spothanis commented Apr 10, 2015

lavalamp commented Apr 10, 2015

markturansky Apr 10, 2015

Choose a reason for hiding this comment

spothanis Apr 10, 2015

Choose a reason for hiding this comment

thockin Apr 10, 2015

Choose a reason for hiding this comment

thockin Apr 10, 2015

Choose a reason for hiding this comment

thockin commented Apr 10, 2015

markturansky commented Apr 10, 2015

thockin commented Apr 15, 2015

pmorie commented Apr 15, 2015

wattsteve commented May 7, 2015

spothanis commented May 7, 2015

wattsteve commented May 7, 2015

spothanis commented May 7, 2015

k8s-bot commented May 22, 2015

erictune commented Jun 1, 2015

k8s-bot commented Jun 1, 2015

thockin Jun 1, 2015

Choose a reason for hiding this comment

spothanis Jun 2, 2015

Choose a reason for hiding this comment

k8s-bot commented Jul 8, 2015

pmorie commented Jul 8, 2015

saad-ali commented Aug 27, 2015

spothanis commented Aug 27, 2015

pmorie commented Aug 27, 2015

spothanis commented Aug 27, 2015

pmorie commented Aug 27, 2015

spothanis commented Aug 27, 2015

pmorie commented Aug 27, 2015

k8s-github-robot commented Aug 27, 2015

bprashanth commented Aug 28, 2015

pmorie commented Aug 28, 2015

bprashanth commented Aug 28, 2015

saad-ali commented Aug 28, 2015

bprashanth commented Aug 28, 2015

spothanis commented Aug 28, 2015

liggitt commented Aug 28, 2015

bprashanth commented Aug 28, 2015

pmorie commented Aug 28, 2015

bprashanth commented Aug 28, 2015

spothanis commented Aug 28, 2015

wattsteve commented Aug 31, 2015