cmd/serve/restic: add append-only mode

[skriss]

This PR adds an optional append-only mode to the restic serve cmd, similar to the one implemented in restic's rest-server. Specifically, deletes to any resources that are not locks are disallowed. For reference, here are the spots in the rest-server handlers where this is used:

https://github.com/restic/rest-server/blob/master/handlers.go#L233
https://github.com/restic/rest-server/blob/master/handlers.go#L527

[ncw]

Rclone will still allow objects to be overwritten. It used to have a check to disallow that but we decided it wasn't necessary. Do you need that for an append only mode?

[skriss]

Restic never overwrites existing objects so I think we should be fine as-is.

[ncw]

Thanks for clarifying that - it looks great :-)

[fd0]

Oh, thanks a lot for implementing this feature! For it to work correctly, we need to check that no files are overwritten. It's something that restic itself won't do, but it's relevant to the situation where the append-only mode is needed. I've described the scenario here: restic/caddy#2 (comment) Attackers may very well talk to rclone directly and instruct it to overwrite files.

When attackers gained access to a server, they learn the credentials needed to access rclone (e.g. an SSH key or username/password for the REST server). Usually, this is enough for them to empty the repo and destroy all backups (after all, deleting files in the repo is needed for restic prune to work). When the append-only mode is used, attackers should not be able to delete any files in the repo. That's already implemented. However, with the current code, it's possible for attackers to overwrite existing files, for example with empty files. That will still destroy backups, which should be prevented in append-only mode. Hm.

The REST server we've implemented does not have such a check in the SaveBlobs function any more because it opens local files with O_EXCL. When the file already exists, the call to os.Open fails, so it's not possible to overwrite files.

We need to add this test to rclone, but it should only be active in append-only mode. Is there a way (with the FS implementation of rclone) to maybe simulate O_EXCL?

I've opened #2195 to track this issue.