Backend: Amazon Glacier

[osiloke]

I would like to use restic to do nightly backups for long term storage and glacier seems to be a cheap solution. I'm about to dive into the current s3 backend and see how i can transmorph that into a glacier backend. I was wondering if anyone had any plans for this or any suggestions on how i could go about it.

Thanks

[fd0]

Hi, thanks for the suggestion. I've already thought about this and I think that Glacier is unsuitable (as far as I understood the concept) for restic. At the moment, restic holds all the state in at the backend, which means it downloads quite some data before starting a backup. For Glacier, this means waiting 3-5 hours to start the backup, according to the FAQ:

Q: How can I retrieve data from the service?

You can download data directly from the service using the service’s REST API. When you make a request to retrieve data from Glacier, you initiate a retrieval job. Once the retrieval job completes, your data will be available to download for 24 hours. Retrieval jobs typically complete within 3-5 hours.

I think it will require many changes in restic's repository architecture before it is feasible to try and implement a backend for Glacier.

What do you think?

[osiloke]

I was thinking of a proxy type solution, where a local backup is kept as well as pushed to the glacier backend. You would not need to download a backup everytime but only when you loose the local data. It's kind of a worst case scenario. For example, being hit by a "ransomware" which encrypts all local backups (somehow), maybe if a user mounts a backup folder on an affected system.

[fd0]

I was thinking of a proxy type solution, where a local backup is kept as well as pushed to the glacier backend.

You can easily achieve this already by create a local repository, using this for the backup, and then use a Glacier client to copy the files to Amazon.

[osiloke]

Yes that was obvious, i was on a path of over engineering. I'll do that instead. Thanks

[rektide]

I've already thought about this and I think that Glacier is unsuitable (as far as I understood the concept) for restic. At the moment, restic holds all the state in at the backend, which means it downloads quite some data before starting a backup.

Would obviously be a lot of complexity, but I could see a hybrid solution being a possible: use regular S3 to hold state, but send contents to Glacier. Just a thought.

[buzzlawless]

Glacier now offers expedited retrieval (1-5 minutes), which makes it a more appealing option for restic.

[andreasnuesslein]

you can also put a policy on any s3 bucket, telling it to put certain objects into glacier after a period of days.
you could maybe put anything that restic does not have to access regularly (assuming index files and such?, so: all the data objects) into that policy.

https://aws.amazon.com/de/blogs/aws/archive-s3-to-glacier/

[bscott]

You could just have the first bucket with a policy for replication of a copy of all objects to another bucket that is glacier enabled.