support azure file snapshot

[andyzhangx]

Currently azure cloud provide only supports azure disk snapshot, and azure file plugin is also a common storage layer which could provide multiple read/write.
Azure file snapshot doc:
https://docs.microsoft.com/en-us/azure/storage/files/storage-snapshots-files

@rosskukulinski

I could also do the code review work of all azure part since I have been working on k8s on azure storage for a long time.

[ncdc]

@andyzhangx what would the mechanics be around restores? With Azure managed disks, for example, we create a new managed disk from a snapshot and associate that with the Kubernetes PersistentVolume. Could we do something similar with Azure files - can we create a new "share" from a snapshot and associate that with a PV? Or how would you envision this working? Thanks!

[andyzhangx]

Yes, azure file pvc is like azure disk pvc. It's using different protocol and support multiple readwrite.
And its apiVersion requires >= 2017-04-17, see RESET api here:
https://docs.microsoft.com/en-us/rest/api/storageservices/snapshot-share

[moebius87]

Hi @ncdc , is there any good news about this issue? Thank you!

[skriss]

@moebius87 nobody has picked this up to work on it, but it sounds like there's a good chance that a plugin could be developed for this. I'm happy to do a little R&D and provide some direction if someone wants to work on it!

[andyzhangx]

Update: we are already implementing azure file snapshot functionality in https://github.com/kubernetes-sigs/azurefile-csi-driver, currently azure file does not provide a restore sdk for this functionality, and we are expecting azure file team to provide a v2 sdk to cover this functionality. I think we could wait until that restore function is released, stay tuned.

[skriss]

@andyzhangx any update on the restore SDK? There have been several folks inquiring about Azure File support lately, and they're having to default to using restic since we don't have a snapshotter plugin.

[andyzhangx]

@andyzhangx any update on the restore SDK? There have been several folks inquiring about Azure File support lately, and they're having to default to using restic since we don't have a snapshotter plugin.

No update from azure file team now, I will let you know when that feature is released. @skriss

[pimjansen]

Also really need this! There is no way to properly snapshot/backup my azurefile at this point

[fredgate]

Yes, it's a big lack.

[RuBiCK]

Any update on this? I really need this feature and I would like to know if it's on the roadmap

[RuBiCK]

For those who are looking at velero for solve this issue, there is a workaround: Velero Restic plugin so restic will be in charge to backup file shares just with a POD annotation.
I haven't tested yet. I'll let you know in the few weeks when I test it

[kwaazaar]

To use restic for a specific pod, add an annotation to specify which volumes to include:

backup.velero.io/backup-volumes: "data"

Velero restore-process remains the same.

I prefer this opt-in model. Read more here.

[felfa01]

Do you have an update on this much sought-after feature? Would be great to understand the timeline for when this becomes available.

[IQNeoXen]

Any news on this? Would be really nice to have this implemented

[rajeevpnair]

Any update on this ? we are really looking for an option to do csi snapshot backups for azure files..

[reasonerjt]

@ywk253100 probably this can be handled by CSI plugin? Could you verify?

[andyzhangx]

@ywk253100 probably this can be handled by CSI plugin? Could you verify?

azure file csi driver only supports snapshot, while does not support restore since current azure file API only supports in-place restore, not CSI spec compliant way, here is the tracking issue: kubernetes-sigs/azurefile-csi-driver#136

[reasonerjt]

@andyzhangx
But I believe eventually this has to be resolved in 1.27? Could you share some insights?

[davidkarlsen]

It's been a while - any update @andyzhangx?
And would this work when you use:

parameters:
  protocol: nfs

in the storage-class?

Is azure-files still a thing or being killed in favour of NetApp given that there is no movement?

[andyzhangx]

It's been a while - any update @andyzhangx? And would this work when you use:

parameters:
  protocol: nfs

in the storage-class?

Is azure-files still a thing or being killed in favour of NetApp given that there is no movement?

@davidkarlsen there is a way to support smb file share restore using azcopy, here is a draft PR: kubernetes-sigs/azurefile-csi-driver#1466, and for nfs file share, the snapshot feature is only supported on Azure canary region now

[draghuram]

Just a quick note to add that at CloudCasa, we implemented a work around where we used Azure APIs to directly read from the snapshot, instead of depending on CSI. This is used in couple of places: For backups, we take CSI snapshot first and then using our work around, we get the data from snapshot and copy to S3. Similar logic is used for restores as well. So for end users, it looks seamless though it would have been much better had CSI "volume from snapshot" is implemented.

From what I hear, the problem is not with the CSI driver but with the underlying Azure Files functionality itself. If that is the case, I doubt we will have this feature implemented any time soon, though I could be wrong.

[maulik13]

It's been a while - any update @andyzhangx? And would this work when you use:

parameters:
  protocol: nfs

in the storage-class?
Is azure-files still a thing or being killed in favour of NetApp given that there is no movement?

@davidkarlsen there is a way to support smb file share restore using azcopy, here is a draft PR: kubernetes-sigs/azurefile-csi-driver#1466, and for nfs file share, the snapshot feature is only supported on Azure canary region now

Is the mentioned PR going to be merged any time soon?

[andyzhangx]

It's been a while - any update @andyzhangx? And would this work when you use:

parameters:
  protocol: nfs

in the storage-class?
Is azure-files still a thing or being killed in favour of NetApp given that there is no movement?

@davidkarlsen there is a way to support smb file share restore using azcopy, here is a draft PR: kubernetes-sigs/azurefile-csi-driver#1466, and for nfs file share, the snapshot feature is only supported on Azure canary region now

Is the mentioned PR going to be merged any time soon?

@maulik13 we are still reviewing the azcopy design.

[felfa01]

Given that AzureFile NFS supports Snapshots, is this something that can be supported from Velero side?

[draghuram]

Velero can even today take snapshot of AzureFile PVCs. The problem is that restore cannot be supported because AzureFile CSI driver doesn't support creation of a PVC from snapshot. For the same reason, snapshot data mover backups won't work either. We, at CloudCasa, have added workaround by directly interacting with Azure Portal and thus, supporting both.