Investigate Kale alongside the Generic Snapshot support

[sylus]

There is a P.R. I am keen to test out that will allow Kale to snapshot our Notebook volumes:

kubeflow-kale/kale#217

Basically if taking snapshots is possible with the current CSI driver, the user will be able to select "Use this notebook's volumes" or "Clone Notebook Volume" and will create a snapshot of the notebook's PVC(s).

What is neat is that we run Velero in our cluster which does support snapshots with its CSI Driver:

https://github.com/vmware-tanzu/velero/blob/main/design/Implemented/csi-snapshots.md

What this means is with Minimal effort we could have this working ^_^

Even easier though is we can use the Azure CSI Driver which has snapshot func as well:

https://github.com/kubernetes-sigs/azurefile-csi-driver/tree/master/deploy/example/snapshot

[sylus]

Looks like will need Velero to fully handle this for the moment:

vmware-tanzu/velero#3151

[sylus]

We will need to enable the CSI support but is just a flag to our Helm chart.

https://velero.io/blog/csi-integration/

[davidspek]

Hi @sylus, thanks for taking interest in my PR. Fun to get in touch with fellow Canadians. So as you can see the PR is a bit stale as I have my time split between many things and this is my first programming work. If you like we can discuss it further on Slack and I can get you up to speed where I left off.

Creating a snapshot of a notebook works, along with restoring a notebook from a snapshot. Other than the obvious things such as UUIDs for the snapshot names and such, the functional part that is missing is in the step where Kale generates the Pipeline code. Kale currently writes an annotation with a rok/url, and Rok then scans for this and creates the required PVC. Kale would need to be changed so that when using the CSI driver, it writes a data_source so that KFP creates a PVC from the given snapshot. https://kubeflow-pipelines.readthedocs.io/en/stable/source/kfp.dsl.html#kfp.dsl.VolumeOp

What might also interest you is the upcoming Volume Manager UI that Arrikto is going to open source hopefully next week. https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&cad=rja&uact=8&ved=2ahUKEwj4narnnrDuAhXJy6QKHe6nAqsQFjAAegQIBRAC&url=https%3A%2F%2Fjournal.arrikto.com%2Fdemocratizing-the-use-of-pvcs-with-the-introduction-of-a-volume-manager-ui-a5bf616570d3&usg=AOvVaw0AwvCAMshONsrCtBeuj1l8

I plan to reuse some code from the Kale PR to add snapshot creation/management and notebook restoration to the Volume Manager UI.