Volume Snapshots in Kubernetes
This document describes current state of Volume Snapshot support in Kubernetes provided by external controller and provisioner. Familiarity with Persistent Volumes, Persistent Volume Claims and Dynamic Provisioning is recommended.
Many storage systems provide the ability to create "snapshots" of a persistent volume to protect against data loss. The external snapshot controller and provisioner provide means to use the feature in Kubernetes cluster and handle the volume snapshots through Kubernetes API.
- Create snapshot of a
PersistentVolumebound to a
- List existing
- Delete existing
- Create a new
PersistentVolumefrom an existing
- Amazon EBS
- GCE PD
- OpenStack Cinder
Lifecycle of a Volume Snapshot and Volume Snapshot Data
Prerequisites for using the snapshotting features are described in sections "Persistent Volume Claim and Persistent Volume" and "Snapshot Promoter".
Persistent Volume Claim and Persistent Volume
The user already created a Persistent Volume Claim that is bound to a Persistent Volume. The Persistent Volume type must be one of the snaphot supported Persistent Volume types.
An admin created a Storage Class like the one shown below:
kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: snapshot-promoter provisioner: volumesnapshot.external-storage.k8s.io/snapshot-promoter
Such Storage Class is necessary for restoring a Persistent Volume from already created Volume Snapshot and Volume Snapshot Data.
VolumeSnapshot contains a spec and status, which is the specification and status of the Volume Snapshot.
apiVersion: volumesnapshot.external-storage.k8s.io/v1 kind: VolumeSnapshot metadata: name: snapshot-demo spec: persistentVolumeClaimName: ebs-pvc
persistentVolumeClaimName: name of the Persistent Volume Claim that is bound to a Persistent Volume. This particular Persistent Volume will be snapshotted.
Volume Snapshot Data is automatically created based on the Volume Snapshot. Relationship between Volume Snapshot and Volume Snapshot Data is similar to the relationship between Persistent Volume Claim and Persistent Volume.
Depending on the Persistent Volume type the operation might go through several phases which are reflected by the Volume Snapshot status:
- The new Volume Snapshot object is created.
- The controller starts the snapshot operation: the snapshotted Persistent Volume might need to be frozen and the applications paused.
- The storage system finishes creating the snapshot (the snapshot is "cut") and the snapshotted Persistent Volume might return to normal operation. The snapshot itself is not ready yet. The last status condition is of
Pendingtype with status value "True". A new VolumeSnapshotData object is created to represent the actual snapshot.
- The newly created snapshot is completed and ready to use. The last status condition is of
Readytype with status value "True"
- It is the user's responsibility to ensure the data consistency (stop the pod/application, flush caches, freeze the filesystem, ...).
- In case of error in any of the steps the Volume Snapshot status is appended with an
A Volume Snapshot status can be displayed as shown below:
$ kubectl get volumesnapshot -o yaml
apiVersion: volumesnapshot.external-storage.k8s.io/v1 kind: VolumeSnapshot metadata: clusterName: "" creationTimestamp: 2017-09-19T13:58:28Z generation: 0 labels: Timestamp: "1505829508178510973" name: snapshot-demo namespace: default resourceVersion: "780" selfLink: /apis/volumesnapshot.external-storage.k8s.io/v1/namespaces/default/volumesnapshots/snapshot-demo uid: 9cc5da57-9d42-11e7-9b25-90b11c132b3f spec: persistentVolumeClaimName: ebs-pvc snapshotDataName: k8s-volume-snapshot-9cc8813e-9d42-11e7-8bed-90b11c132b3f status: conditions: - lastTransitionTime: null message: Snapshot created successfully reason: "" status: "True" type: Ready creationTimestamp: null
In order to restore a Persistent Volume from a Volume Snapshot a user creates the following Persistent Volume Claim:
apiVersion: v1 kind: PersistentVolumeClaim metadata: name: snapshot-pv-provisioning-demo annotations: snapshot.alpha.kubernetes.io/snapshot: snapshot-demo spec: storageClassName: snapshot-promoter
snapshot.alpha.kubernetes.io/snapshot: the name of the Volume Snapshot that will be restored.
storageClassName: Storage Class created by admin for restoring Volume Snapshots.
A Persistent Volume will be created and bound to the Persistent Volume Claim. The process may take several minutes depending on the Persistent Volume Type.
A Volume Snapshot
snapshot-demo can be deleted as shown below:
$ kubectl delete volumesnapshot/snapshot-demo
The Volume Snapshot Data that are bound to the Volume Snapshot are also automatically deleted.
Managing Snapshot Users
Depending on the cluster configuration it might be necessary to allow non-admin users to manipulate the VolumeSnapshot objects on the API server. This might be done by creating a ClusterRole bound to a particular user or group.
Assume the user 'alice' needs to be able to work with snapshots in the cluster. The cluster admin needs to define a new ClusterRole.
apiVersion: v1 kind: ClusterRole metatadata: name: volumesnapshot-admin rules: - apiGroups: - "volumesnapshot.external-storage.k8s.io" attributeRestrictions: null resources: - volumesnapshots verbs: - create - delete - deletecollection - get - list - patch - update - watch
Now the cluster role has to be bound to the user 'alice' by creating a ClusterRole binding object.
apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: volumesnapsot-admin roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: volumesnapshot-admin subjects: - kind: User name: alice
This is only an example of API access configuration. Note the VolumeSnapshot objects behave just like any other Kubernetes API objects. Please refer to the API access control documentation for complete guide on managing the API RBAC.