-
Notifications
You must be signed in to change notification settings - Fork 38.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
support storage class in Ceph RBD volume #31251
Merged
Merged
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -121,6 +121,32 @@ parameters: | |
* `type`: [VolumeType](http://docs.openstack.org/admin-guide/dashboard-manage-volumes.html) created in Cinder. Default is empty. | ||
* `availability`: Availability Zone. Default is empty. | ||
|
||
#### Ceph RBD | ||
|
||
```yaml | ||
apiVersion: extensions/v1beta1 | ||
kind: StorageClass | ||
metadata: | ||
name: fast | ||
provisioner: kubernetes.io/rbd | ||
parameters: | ||
monitors: 10.16.153.105:6789 | ||
adminID: kube | ||
adminSecretName: ceph-secret | ||
adminSecretNamespace: kube-system | ||
pool: kube | ||
userId: kube | ||
secretName: ceph-secret-user | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
``` | ||
|
||
* `monitors`: Ceph monitors, comma delimited | ||
* `adminID`: Ceph client ID that is capable of creating images in the pool. Default is "admin" | ||
* `adminSecret`: Secret Name for `adminID` | ||
* `adminSecretNamespace`: The namespace for `adminSecret`. Default is "default" | ||
* `pool`: Ceph RBD pool. Default is "rbd" | ||
* `userId`: Ceph client ID that is used to map the RBD image. Default is the same as `adminID` | ||
* `secretName`: The name of Ceph Secret. It must exist in the same namespace as PVCs. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. note that this is the "secret used to map the RBD image" |
||
|
||
### User provisioning requests | ||
|
||
Users request dynamically provisioned storage by including a storage class in their `PersistentVolumeClaim`. | ||
|
@@ -152,6 +178,7 @@ In the future, the storage class may remain in an annotation or become a field o | |
|
||
### Sample output | ||
|
||
#### GCE | ||
This example uses GCE but any provisioner would follow the same flow. | ||
|
||
First we note there are no Persistent Volumes in the cluster. After creating a storage class and a claim including that storage class, we see a new PV is created | ||
|
@@ -184,6 +211,73 @@ $ kubectl get pv | |
|
||
``` | ||
|
||
|
||
#### Ceph RBD | ||
|
||
First create Ceph admin's Secret in the system namespace. Here the Secret is created in `kube-system`: | ||
|
||
``` | ||
$ kubectl create -f examples/experimental/persistent-volume-provisioning/rbd/ceph-secret-admin.yaml --namespace=kube-system | ||
``` | ||
|
||
Then create RBD Storage Class: | ||
|
||
``` | ||
$ kubectl create -f examples/experimental/persistent-volume-provisioning/rbd/rbd-storage-class.yaml | ||
``` | ||
|
||
Before creating PVC in user's namespace (e.g. myns), make sure the Ceph user's Secret exists, if not, create the Secret: | ||
|
||
``` | ||
$ kubectl create -f examples/experimental/persistent-volume-provisioning/rbd/ceph-secret-user.yaml --namespace=myns | ||
``` | ||
Now create a PVC in user's namespace (e.g. myns): | ||
|
||
``` | ||
$ kubectl create -f examples/experimental/persistent-volume-provisioning/claim1.json --namespace=myns | ||
``` | ||
|
||
Check the PV and PVC are created: | ||
``` | ||
$ kubectl describe pvc --namespace=myns | ||
Name: claim1 | ||
Namespace: myns | ||
Status: Bound | ||
Volume: pvc-1cfa23b3-664b-11e6-9eb9-90b11c09520d | ||
Labels: <none> | ||
Capacity: 3Gi | ||
Access Modes: RWO | ||
No events. | ||
|
||
$ kubectl describe pv | ||
Name: pvc-1cfa23b3-664b-11e6-9eb9-90b11c09520d | ||
Labels: <none> | ||
Status: Bound | ||
Claim: myns/claim1 | ||
Reclaim Policy: Delete | ||
Access Modes: RWO | ||
Capacity: 3Gi | ||
Message: | ||
Source: | ||
Type: RBD (a Rados Block Device mount on the host that shares a pod's lifetime) | ||
CephMonitors: [10.16.153.105:6789] | ||
RBDImage: kubernetes-dynamic-pvc-1cfb1862-664b-11e6-9a5d-90b11c09520d | ||
FSType: | ||
RBDPool: kube | ||
RadosUser: kube | ||
Keyring: /etc/ceph/keyring | ||
SecretRef: &{ceph-secret-user} | ||
ReadOnly: false | ||
No events. | ||
``` | ||
|
||
Create a Pod to use the PVC: | ||
|
||
``` | ||
$ kubectl create -f examples/experimental/persistent-volume-provisioning/rbd/pod.yaml --namespace=myns | ||
``` | ||
|
||
|
||
<!-- BEGIN MUNGE: GENERATED_ANALYTICS --> | ||
[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/examples/experimental/persistent-volume-provisioning/README.md?pixel)]() | ||
<!-- END MUNGE: GENERATED_ANALYTICS --> |
6 changes: 6 additions & 0 deletions
6
examples/experimental/persistent-volume-provisioning/rbd/ceph-secret-admin.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
apiVersion: v1 | ||
kind: Secret | ||
metadata: | ||
name: ceph-secret-admin | ||
data: | ||
key: QVFEQ1pMdFhPUnQrSmhBQUFYaERWNHJsZ3BsMmNjcDR6RFZST0E9PQ== |
6 changes: 6 additions & 0 deletions
6
examples/experimental/persistent-volume-provisioning/rbd/ceph-secret-user.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
apiVersion: v1 | ||
kind: Secret | ||
metadata: | ||
name: ceph-secret-user | ||
data: | ||
key: QVFBTWdYaFZ3QkNlRGhBQTlubFBhRnlmVVNhdEdENGRyRldEdlE9PQ== |
23 changes: 23 additions & 0 deletions
23
examples/experimental/persistent-volume-provisioning/rbd/pod.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
apiVersion: v1 | ||
kind: ReplicationController | ||
metadata: | ||
name: server | ||
spec: | ||
replicas: 1 | ||
selector: | ||
role: server | ||
template: | ||
metadata: | ||
labels: | ||
role: server | ||
spec: | ||
containers: | ||
- name: server | ||
image: nginx | ||
volumeMounts: | ||
- mountPath: /var/lib/www/html | ||
name: mypvc | ||
volumes: | ||
- name: mypvc | ||
persistentVolumeClaim: | ||
claimName: claim1 |
14 changes: 14 additions & 0 deletions
14
examples/experimental/persistent-volume-provisioning/rbd/rbd-storage-class.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
apiVersion: extensions/v1beta1 | ||
kind: StorageClass | ||
metadata: | ||
name: slow | ||
provisioner: kubernetes.io/rbd | ||
parameters: | ||
monitors: 10.16.153.105:6789 | ||
adminID: admin | ||
adminSecretName: ceph-secret-admin | ||
adminSecretNamespace: "kube-system" | ||
pool: kube | ||
userId: kube | ||
secretName: ceph-secret-user | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
adminId
, to followuserId
below. Or the other way around.