--b2-versions doesn't work for crypt backed by B2

[jikamens]

Hidden file versions are not listed when I do rclone ls --b2-versions on a crypt bucket that is backed by a B2 bucket. This is highly inconvenient, to say the least. :-/

[jikamens]

Does rclone cleanup on a B2-backed crypt bucket even work? Not sure.

[ncw]

I'm not 100% sure how to fix this! Perhaps by getting the crypt remote to recognise the suffixes added for the versions.

Does rclone cleanup on a B2-backed crypt bucket even work? Not sure.

I'm guessing yes as the crypt remote wraps the cleanup call. If it doesn't, please make another issue!

[keithgh1]

Info from dupe #1632

When you have B2 "lifecycle settings" set to include versioning, and you upload a new file in the same directory, with the same filename, then the B2 server has more than one version of the file with different timestamps.

The documented behavior is here

https://www.backblaze.com/b2/docs/file_versions.html

and

https://www.backblaze.com/b2/docs/lifecycle_rules.html (describes which versions to keep etc)

Due to the filenames being encrypted locally by rclone prior to uploading, these additional versions now available on the server causes the associated filenames to be a mix of ciphertext and a plaintext date/time stamp.

When rclone attempts to decrypt this now-modified filename, the decryption fails and rclone throws an error (visible with more verbose debugging enabled) saying that the file has an "invalid filename" and skips processing this file.

Ideally, rclone could identify when a B2 encrypted remote, using filename encryption, with B2 versioning enabled server-side, sees an "invalid filename", could have an additional decryption attempt, by properly parsing the now-modified filename into two parts. The first part, the original encrypted filename, could run through the decryption process normally. The second part, the date/time stamp, would be appended onto the original plaintext filename (result of part1 decryption)

This would allow rclone filename encryption and B2 versioning to work together.

This issue is being filed per ncw. Some discussion is in this thread https://forum.rclone.org/t/backblaze-b2-humming-along-as-a-replacement-for-acd/2580/25

[ivlis]

That would be an extremely useful feature for b2, since it allows a versioned backup with zero cost from the user-side.

[normsland]

Would be great to get this implemented as b2 does not support server side move or copy. So we can't use the --backup-dir switch.

[cowwoc]

This feature, or lack thereof, is a very very big deal to me (the practical implication is data loss in case I want to recover an older version of my data).

Can we expect a fix in the short-term? Or should I consider drastic steps like disabling encryption?

Also, does this really affect all crypt configuratons? Or does it only affect filename encryption?

I would be okay losing filename encryption as a workaround so long as the actual contents are encrypted and I get versioning. Does that work?

[cowwoc]

I just tested it. It looks like versioning works fine if filename encryption is disabled.

[ncw]

I just tested it. It looks like versioning works fine if filename encryption is disabled.

Yes that is a reasonable work-around I think.

[keithgh1]

Yes, the filename encryption is exactly the problem that this bug covers. See my previous message above. I did describe a potential fix above --- whether that's the most prudent one, I have no idea.

Thanks