Skip to content
Shell script for creating, encrypting, splitting and pushing archives to AWS Glacier.
Shell
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
LICENSE
README.md
aws-glacier-backup.sh

README.md

AWS Glacier Backup

Description

Shell script for creating, encrypting, splitting and pushing archives to AWS Glacier.

This will create a directory at /opt/aws-glacier-backup/COMPLETED_JOBS with the name of your archive concatenated with the date and time. Inside will be a manifest of all files in the archive and two json files with Glacier related metadata.

Prerequisites

jq

https://stedolan.github.io/jq/

treehash

https://github.com/jdswinbank/treehash

PyPi

https://pypi.org/

AWS CLI

https://aws.amazon.com/cli/

GnuPG

https://www.gnupg.org/

bc

https://www.gnu.org/software/bc/

Additionally, this scripts assumes you have already properly configured cli access to AWS Glacier and created a vault for this purpose.

Variables

Variable Description Default Value
sourceDir Base directory none
archiveDir Directory containing items to be archived none
dateArchiveName Name of created tar.gz archive concatenated from archiveDir and the current date none
chunkSize Largest allowable single object size for AWS Glacier 4294967296
chunkSizeGiB Calculated size of $chunkSize in gibibytes $chunkSize/1073741824
chunkSizeMiB Calculated size of $chunkSize in mibibytes $chunkSize/1048576
archiveSize Resultant size of encrypted tar.gz archive none
vaultName Name of AWS Glacier vault where uploads will be targeted none
gpgRecipient Valid email or key id that the archive will be encrypted with none
TREEHASH Calculated SHA256 tree hash for validating upload parts none
UPLOADID AWS Glacier upload ID returned as a json value none
ARCHIVEID AWS Glacier archive ID returned as a json value none
ARCHIVEID_LEN Character length of ARCHIVEID; valid ARCHIVEID is 138 characters long none

There are some other variables, but they are used for mundane stuff like calculating ranges.

Syntax

./aws-glacier-backup.sh /source/directory/path target_directory

Example

./aws-glacier-backup.sh /mnt/freenas/projects VisioDrawings

will create the following files:

Archive Manifest JSON
VisioDrawings_03-20-2018_1215.tar.gz manifest_VisioDrawings_03-20-2018_1215.txt meta_upload_init_VisioDrawings_03-20-2018_1215.json
meta_upload_complete_VisioDrawings_03-20-2018_1215.json

The archive will be uploaded to Glacier and deleted locally. The manifest and json files are stored in /opt/aws-glacier-backup/COMPLETED_JOBS/VisioDrawing_03-20-2018_1215)

Written By

David Hunter, 2018

You can’t perform that action at this time.