Request: Periodic discard/trim (SCSI unmap) vs. online discard

[wonderland]

Describe the solution you'd like
Trident already enables (and documents) use of the "discard" mount option to enable block reclamation on ontap-san and ontap-san-economy drivers. However, OS vendors such as Red Hat recommend to run block reclamation as a periodic task ("trim" command) rather than online discard via the mount options.
Could Trident implement a periodic trim operation on all LUNs? Or trigger this during each volume mount to at least run it from time to time? This would allows to stay in line with OS vendor recommendations and still use this efficiency feature.

Describe alternatives you've considered
Online block discard via mount option in Storage Class. This is not recommended, for example Red Hat states "Red Hat recommends that you use batch or periodic discard."
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/managing_file_systems/discarding-unused-blocks_managing-file-systems

[seijitanabe]

My Customer is also facing this issue.
When enabling discard option in the StorageClass, I/O stoppages occur frequently on the volumes that are frequently written and deleted 60KB files. (When disabling this option, it does not happen at all.)

It should be mentioned in the Trident documentation that OS vendors don't recommend it and that discard option may not always be appropriate.

[gnarl]

@wonderland and @seijitanabe,

Initial thoughts are that we will update the Trident documentation for the user to decide how to implement the discard option. Setting the periodic discard option is a host level operation systemctl enable --now fstrim.timer that is outside of what Trident would typically manage. The main reason for this is that many customers use configuration management tools for their Kubernetes worker nodes and Trident should never be in conflict with these tools. Another reason being the history of opinionated configuration and changes to service managers like systemd across distros and time.

@seijitanabe, you can recommend that your customer configure their worker nodes to use periodic discard. Thanks for outlining the use case where the inline discard is problematic.

[wonderland]

I agree that updated docs would be a great starting point.

I also agree that enabling systemd services is a bit out of scope for Trident. But I realize my wording was a bit misleading, I tried to suggest that Trident triggers a periodic batch discard, not that Trident should enable the period discard of the OS (e.g. fstrim.timer). Yes, that essentially means that Trident would re-implement what fstrim.timer can already do, which is not optimal. But:

• Worker nodes tend to be treated as immutable, where uses cannot easily make changes to system services. If fstrim.timer is not enabled, users currently have no choice in such situations.
• Many OS have fstrim.timer enabled/scheduled by default (typically for Monday morning). However, with K8s we tend to have a larg(er) number of PVC/LUNs attached. That raises concern about performance impact of running discard operations for all of them at the same time.
• Weekly schedule for discard (as is the default for fstrim.timer) might not be a perfect match for a much more agile platform such as K8s

If Trident would periodically trigger a discard operations, in a way that ensures it is not run on all LUNs at the same time, that should result in much smoother operations. It would not even have to be truly periodically but could be "attached" to certain operations where Trident interacts with that specific PVC anyway.

Happy to hear your thoughts on this.

[seijitanabe]

@gnarl Thanks. My customer has already configured their worker nodes to use periodic discard. I agree with this update to the docs.

[gnarl]

@wonderland and @seijitanabe,

Thanks for the feedback. I'll talk to the team and see what they think. If they need additional information we know how to get in touch with you.