vfs: option to cache metadata only

[ivandeex]

What is your current rclone version (output from rclone version)?

1.54

What problem are you are trying to solve?

Requested by @Dibbyo456 in #4097 (comment)

Can we cache metadata only in VFS?

How do you think rclone should be changed to solve that?

Proposed by @ncw in https://forum.rclone.org/t/how-to-cache-only-metadata-on-cache-remote/14846/7

• Enable async writeback in a secure way
• Enable --vfs-cache-mode full not to download the entire file immediately
• Persistent metadata caching (needs a bit more work but isn't nearly as tricky!)

@ivandeex
As an option, consider new --vfs-cache-mode metadata ??

Informative references

• https://forum.rclone.org/t/how-to-cache-only-metadata-on-cache-remote/14846/5
• https://rclone.org/commands/rclone_mount/#vfs-file-caching
• https://github.com/rclone/rclone/blob/v1.54.1/vfs/file.go

[christianmerges]

What is your current rclone version (output from rclone version)?
1.63 beta

What problem are you are trying to solve?
When the cache disk is running out of space, even after making the drive bigger, uploads are not started to free up space in the cache. When u restart the daemon and remount. Files are being checked, if they have been modified while the daemon was stopped. Only one file at the time is being read from cache disk and compared to the file in the cloud. This makes it really slow.

How do you think rclone should be changed to solve that?
Save cache metadata on separate folder or reserve space for metadata on the cache disk. So the error would resolve on it's own just by refusing reads for the period of time, where there is no space in cache drive left, but after finished uploads and flushing from cache, reads can be cached again.

When mounting the vfs, read as many files from disk in parallels as the variable "transfers" is set. Most cloud providers limit the bandwidth for a single transfer, but with multiple transfers, the checking goes faster and the vfs would mount much faster.

[tionis]

I'm using rclone as life backend, as such the filesystem is mostly immutable and filepaths are hashed.
For this reason a simple metadata cache could increase performance during push immensely.