Skip to content
Permalink
Browse files

csi-manila: added initial support for snapshots (#652)

* csi-snapshotter is now part of the provisioner StatefulSet

* added initial support for snapshots and stubs for volume cloning

* Gopkg.lock: 2 new dependencies

* github.com/golang/protobuf/ptypes/timestamp
* github.com/gophercloud/gophercloud/openstack/sharedfilesystems/v2/snapshots

* updated docs

* updated examples

* using openstack resource IDs as volume/snapshot IDs; added source vol ID validation
  • Loading branch information...
gman0 authored and k8s-ci-robot committed Jul 16, 2019
1 parent f53dc1e commit 9fa54fa1e54424accec0fe9a74391ca021b106e0

Some generated files are not rendered by default. Learn more.

@@ -10,7 +10,7 @@ As of writing this document, CSI Manila supports only NFS and CephFS shares. If

1. Create a new file `some-protocol.go` under `pkg/csi/manila/shareadapters`
2. Create a new struct that implements the `ShareAdapter` interface
3. Add a case block in `getShareAdapter()` function in `pkg/csi/manila/util.go`. The condition string must match one of Manila's supported share protocols.
3. Add a case block in `getShareAdapter()` function in `pkg/csi/manila/adapters.go`. The condition string must match one of Manila's supported share protocols.
4. Add the protocol name to the `matches` expression (regexp syntax) inside `ControllerVolumeContext.Protocol` field tags in `pkg/csi/manila/options/shareoptions.go`. Again, it must match one of Manila's supported share protocols.
5. Update the docs in `docs/using-manila-csi-plugin.md`, namely any parameters that the protocol or node plugin may use. There's also a dedicated section "Share protocols support matrix" at the bottom of the document which needs to be updated: name of the share protocol, link to the proxy'd CSI driver and its supported version(s).

@@ -22,7 +22,7 @@ Usually, shares / share adapters offer a set of options which users may want to

**Controller Service:**
* `CREATE_DELETE_VOLUME`
* ~~`CREATE_DELETE_SNAPSHOT`~~ planned as a part of GSoC 2019
* `CREATE_DELETE_SNAPSHOT` (snapshotting CephFS shares is not supported yet - planned as a part of GSoC 2019)

Availability Zones are not supported yet.

@@ -1,6 +1,6 @@
# CSI Manila driver

The CSI Manila driver is able to create and mount OpenStack Manila shares.
The CSI Manila driver is able to create and mount OpenStack Manila shares. Snapshots and recovering shares from snapshots is supported as well (support for CephFS snapshots will be added soon).

###### Table of contents

@@ -67,7 +67,7 @@ A single instance of the driver may serve only a single Manila share protocol. T

### Kubernetes 1.13+

Required feature gates: `CSIDriverRegistry`, `CSINodeInfo`
Required feature gates: `CSIDriverRegistry=true`, `CSINodeInfo=true`. Snapshots require `VolumeSnapshotDataSource=true` feature gate.

All Kubernetes YAML manifests are located in `manifests/manila-csi-plugin`.

@@ -1,16 +1,33 @@
## Example CSI Manila CephFS usage
## General notes before continuing

1. Deploy CSI CephFS driver
2. Deploy CSI Manila driver with `--share-protocol-selector=CEPHFS` and `--fwdendpoint=unix:///csi/csi-cephfsplugin/csi.sock` (or similar, based on your environment)
3. Modify `secrets.yaml` to suite your OpenStack cloud environment. Refer to the _"Secrets, authentication"_ section of CSI Manila docs. You may also use helper scripts from `examples/manila-provisioner` to generate the Secrets manifest.
4. Deploy OpenStack secrets
5. Create a persistent volume:
5.1 **If you want to provision a new share:**
5.1.1 Modify `storageclass.yaml` to reflect your environment. Refer to the _"Controller Service volume parameters"_ section of CSI Manila docs.
5.1.2 Deploy the `csi-manila-cephfs` storage class `storageclass.yaml`
5.1.3 Deploy the persistent volume claim `pvc.yaml`
5.2 **OR you want to use an existing share:**
5.2.1 Modify `preprovisioned-pvc.yaml` to reflect your environment. Refer to the _"Node Service volume context"_ section of CSI Manila docs.
5.2.2 Deploy the PV+PVC
6. Deploy `pod.yaml` which creates a Pod that mounts the share you've prepared in the steps above
7. You should see `pod/csicephfs-demo-pod` with status _Running_ soon
1. Make sure you've completed all the steps in `docs/using-manila-csi-plugin.md`: e.g. you've deployed CSI NFS and CSI Manila plugins and CSI Manila is running with `--share-protocol-selector=NFS` and `--fwdendpoint=unix:///csi/csi-nfsplugin/csi.sock` (or similar, based on your environment)
2. Modify `secrets.yaml` to suite your OpenStack cloud environment. Refer to the _"Secrets, authentication"_ section of CSI Manila docs. You may also use helper scripts from `examples/manila-provisioner` to generate the Secrets manifest.
3. The same steps apply to all supported Manila share protocols
4. `exec-bash.sh`, `logs.sh` are convenience scripts for debugging CSI Manila

## Example CSI Manila usage with NFS shares

```
nfs/
├── dynamic-provisioning/
│   ├── pod.yaml
│   ├── pvc.yaml
│   └── --> storageclass.yaml <--
├── snapshot/
│   ├── pod.yaml
│   ├── snapshotclass.yaml
│   ├── snapshotcreate.yaml
│   └── snapshotrestore.yaml
├── static-provisioning/
│ ├── pod.yaml
│ └── --> preprovisioned-pvc.yaml <--
└── --> secrets.yaml <--
```

Files marked with `--> ... <--` need to be customized.

* `dynamic-provisioning/` : creates a new Manila NFS share and mounts it in a Pod
* `static-provisioning/` : fetches an existing Manila NFS share and mounts it in a Pod
* `snapshot/` : takes a snapshot from a PVC source, restores it into a new share and mounts it in a Pod. Deploy manifests in `dynamic-provisioning/` first

After deploying each example you should see the corresponding Pod with status _Running_ soon.
@@ -1,7 +1,7 @@
apiVersion: v1
kind: Pod
metadata:
name: csicephfs-demo-pod
name: new-nfs-share-pod
spec:
containers:
- name: web-server
@@ -13,5 +13,5 @@ spec:
volumes:
- name: mypvc
persistentVolumeClaim:
claimName: csi-manila-cephfs-pvc
claimName: new-nfs-share-pvc
readOnly: false
@@ -1,12 +1,12 @@
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: csi-manila-cephfs-pvc
name: new-nfs-share-pvc
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Gi
storageClassName: csi-manila-cephfs
storageClassName: csi-manila-nfs

@@ -1,10 +1,11 @@
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: csi-manila-cephfs
name: csi-manila-nfs
provisioner: manila.csi.openstack.org
parameters:
type: cephfstype
# Manila share type
type: default

csi.storage.k8s.io/provisioner-secret-name: csi-manila-secrets
csi.storage.k8s.io/provisioner-secret-namespace: default
@@ -0,0 +1,20 @@
apiVersion: v1
kind: Secret
metadata:
name: csi-manila-secrets
namespace: default
stringData:
# Mandatory
os-authURL: "some-auth-url"
os-region: "some-region"

# Authentication using user credentials
os-userName: "some-username"
os-password: "some-password"
os-projectName: "some-project-name"
os-domainID: "some-domain-id"

# Authentication using trustee credentials
os-trustID: "some-trust-id"
os-trusteeID: "some-trustee-id"
os-trusteePassword: "some-trustee-password"
@@ -0,0 +1,17 @@
apiVersion: v1
kind: Pod
metadata:
name: new-nfs-share-snap-restore-pod
spec:
containers:
- name: web-server
image: nginx
imagePullPolicy: IfNotPresent
volumeMounts:
- name: mypvc
mountPath: /var/lib/www
volumes:
- name: mypvc
persistentVolumeClaim:
claimName: new-nfs-share-snap-restore
readOnly: false
@@ -0,0 +1,8 @@
apiVersion: snapshot.storage.k8s.io/v1alpha1
kind: VolumeSnapshotClass
metadata:
name: csi-manila-snapclass
snapshotter: manila.csi.openstack.org
parameters:
csi.storage.k8s.io/snapshotter-secret-name: manila-devstack-secrets
csi.storage.k8s.io/snapshotter-secret-namespace: default
@@ -0,0 +1,9 @@
apiVersion: snapshot.storage.k8s.io/v1alpha1
kind: VolumeSnapshot
metadata:
name: new-nfs-share-snap
spec:
snapshotClassName: csi-manila-snapclass
source:
name: new-nfs-share-pvc
kind: PersistentVolumeClaim
@@ -0,0 +1,15 @@
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: new-nfs-share-snap-restore
spec:
storageClassName: csi-manila-nfs
dataSource:
name: new-nfs-share-snap
kind: VolumeSnapshot
apiGroup: snapshot.storage.k8s.io
accessModes:
- ReadWriteMany
resources:
requests:
storage: 1Gi
@@ -0,0 +1,17 @@
apiVersion: v1
kind: Pod
metadata:
name: existing-nfs-share-pod
spec:
containers:
- name: web-server
image: nginx
imagePullPolicy: IfNotPresent
volumeMounts:
- name: mypvc
mountPath: /var/lib/www
volumes:
- name: mypvc
persistentVolumeClaim:
claimName: existing-nfs-share-pvc
readOnly: false
@@ -1,17 +1,17 @@
apiVersion: v1
kind: PersistentVolume
metadata:
name: preprovisioned-manila-cephfs-share
name: preprovisioned-nfs-share
labels:
name: preprovisioned-manila-cephfs-share
name: preprovisioned-nfs-share
spec:
accessModes:
- ReadWriteMany
capacity:
storage: 1Gi
csi:
driver: manila.csi.openstack.org
volumeHandle: preprovisioned-manila-cephfs-share
volumeHandle: preprovisioned-nfs-share
nodeStageSecretRef:
name: csi-manila-secrets
namespace: default
@@ -25,7 +25,7 @@ spec:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: csi-manila-cephfs-pvc
name: existing-nfs-share-pvc
spec:
accessModes:
- ReadWriteMany
@@ -36,4 +36,4 @@ spec:
matchExpressions:
- key: name
operator: In
values: ["preprovisioned-manila-cephfs-share"]
values: ["preprovisioned-nfs-share"]

This file was deleted.

@@ -40,6 +40,22 @@ spec:
volumeMounts:
- name: socket-dir
mountPath: /var/lib/kubelet/plugins/csi-nodeplugin-manilaplugin
- name: csi-snapshotter
image: quay.io/k8scsi/csi-snapshotter:v1.1.0
args:
- "--csi-address=$(ADDRESS)"
- "--connection-timeout=15s"
- "--v=5"
- "--leader-election=false"
env:
- name: ADDRESS
value: /var/lib/kubelet/plugins/csi-nodeplugin-manilaplugin/csi.sock
imagePullPolicy: Always
securityContext:
privileged: true
volumeMounts:
- name: socket-dir
mountPath: /var/lib/kubelet/plugins/csi-nodeplugin-manilaplugin
volumes:
- name: socket-dir
hostPath:
@@ -9,6 +9,9 @@ apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: manila-external-provisioner-runner
rules:
- apiGroups: [""]
resources: ["nodes"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["secrets"]
verbs: ["get", "list"]
@@ -17,16 +20,32 @@ rules:
verbs: ["list", "watch", "create", "update", "patch"]
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "create", "delete"]
verbs: ["get", "list", "watch", "create", "update", "delete"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshots"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshotcontents"]
verbs: ["create", "get", "list", "watch", "update", "delete"]
- apiGroups: ["snapshot.storage.k8s.io"]
resources: ["volumesnapshotclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: ["apiextensions.k8s.io"]
resources: ["customresourcedefinitions"]
verbs: ["create"]
- apiGroups: ["csi.storage.k8s.io"]
resources: ["csinodeinfos"]
verbs: ["get", "list", "watch"]
- apiGroups: ["storage.k8s.io"]
resources: ["volumeattachments"]
verbs: ["get", "list", "watch", "update"]


---
kind: ClusterRoleBinding

0 comments on commit 9fa54fa

Please sign in to comment.
You can’t perform that action at this time.