Generalize --b2-versions for other backends

[jrk]

Dropbox, Google Drive, and others support file versions in their APIs. They are less rich than b2's versions (I believe Dropbox's versions are read-only, for example), but accessing them would still be very useful.

[jrk]

FWIW, I would personally like to see the interface be a bit more orthogonal than the --b2-versions support seems to be, so relevant commands (e.g., copy/sync/...) would generically apply to a specified source version, using normal file names, rather than embedding the version information in magic file names.

My current use case: I'd like to record local snapshots of my Dropbox at specific points in time in the past. So: rclone sync --version some-date-or-version-number-string dropbox:/ /local/dropbox/mirror, then snapshot /local/dropbox/mirror, and repeat with the next desired version.

[ncw]

What would you see as an interface? Something like --versions-before 2017-02-03 00:00:00 which would show file versions before that date?

The advantage of the --b2-versions flag is that you can easily discover all the versions you do have.

My current use case: I'd like to record local snapshots of my Dropbox at specific points in time in the past. So: rclone sync --version some-date-or-version-number-string dropbox:/ /local/dropbox/mirror, then snapshot /local/dropbox/mirror, and repeat with the next desired version.

You can do this manually at the moment using --backup-dir which works with most providers

[jrk]

To be clear, I want to record and save snapshots of past versions which Dropbox has already recorded (I'm trying to effectively export my long Dropbox version history to somewhere I control), not just avoid deleting old versions on new Dropbox updates, so --backup-dir isn't actually what I want.

For my use case, and I think perhaps in many cases in general, I think the ideal interface would be something like --version-at-date 2017-02-03 00:00:00 which would copy/sync the whole requested tree as it existed at that point in time. This is not very isomorphic to the underlying API, though, and I believe would require pre-indexing the version history of everything at the source before reconstructing that point in time view to know what revisions to pull of each file. This would be an extremely useful feature of the tool, though, specifically because it is a straightforward human-level thing to want to do which requires a fair amount of legwork to build on top of the raw APIs.

Seeing all versions is also useful, but I'd expect that to be a function of the ls command family.

Footnote:

One caveat I notice in implementing this is that I'm not sure it's currently possible in the v2 Dropbox API to reconstruct version history of deleted files. The information all exists on their backend, but it seems the metadata interface in the v2 API is not currently sufficient to reconstruct anything about deleted files: https://www.dropboxforum.com/t5/API-support/Deleted-files-metadata/m-p/213170#M10825 It was previously possible in the v1 API to see all versions of deleted files, so hopefully it will return.

[austinginder]

Yes this would effectively be time machine. Would be amazing if you could do something like rclone mount b2:backup ~/Desktop/RestoreBackup --time-machine 2017-02-03 and rclone could magically show only the files that existed at that time.

With this there would be no need to use a backup repository unless you were wanting advanced features like data deduplication. You could simply use a cloud provider who supports unlimited revisions, like B2, and restore an entire directory from whenever as needed.

[jplona]

Another scenario where this would be useful:

A cron job syncs my backups to b2 daily. Say one day the disk starts going bad and I sync some garbage to my b2 bucket. I'd like to be able to specify to rclone sync "download the latest as of yesterday" when restoring to the replacement drive.