Web service to proxy streaming uploads to Amazon Glacier via a simple REST API (simple enough to use it with curl on the command line)
JavaScript Erlang Perl Other
Switch branches/tags
Nothing to show
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.


Glacier Proxy

The Amazon Glacier API is quite complex and can only be utilized with custom code. Glacier proxy will take care of all the nitty-gritty details of the Amazon API internally and expose it as a simple to use REST API.

While encapsulating a lot of complexity, Glacier Proxy can still be queried for the status of all ongoing activity.

The goal of Glacier Proxy is to enable scripted uploads, particularly of large streams of data that do not exist on disk. It acts as a single point of contact to initiate new jobs and to keep track of the progress/state of the tasks submitted to it.


Glacier Proxy is still work in progress (as in: the pieces are slowly coming together, doing README-driven development towards the following specs).


The CURL examples assume a Glacier Proxy to be running on the same machine on its default port of 8001.

Query Status

GET /status

JSON Response

          "bytesTotal":4321,  // when the request sent Content-Length header
          "msg":"extended human readable message for the status"},

Upload New Archive

POST /vault/:vault_name?job_id=:user_supplied_unique_reference

JSON Response

 "date:"Sun, 2 Sep 2012 12:00:00 GMT",

Usage Example: Upload an Existing File

curl -X POST --upload-file mypreciousfile \

Usage Example: Upload a Data Stream

In this example we create a data stream as Stdin to CURL that represents a compressed and encrypted version of mypreciousfile.

gzip myfile | \
  openssl enc -aes-256-cbc -salt -pass pass:secretkey | \
  curl -X POST --upload-file - \

To be able to verify the content sha1 in the proxy response, we can caclulate a SHA1 on the fly by making use of Bash process substitution like this:

gzip myfile | \
  openssl enc -aes-256-cbc -salt -pass pass:secretkey | \
  tee >(shasum > shasum.txt) | \
  curl -X POST --upload-file - \

Afterwards we can compare the SHA in shasum.txt to the one in the JSON response to make absolutely sure that the file has been uploaded correctly.