Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Rotate, compress, and upload a log to S3

branch: master

Fetching latest commit…

Cannot retrieve the latest commit at this time

README
aws_logmover.py
Mike Babineau <michael.babineau@gmail.com>
http://github.com/mbabineau/aws_logmover/
------------------
Description:
Script for moving logs files into and out of S3 using SQS messages for reference.

aws_logmover supports two modes:
push - Renames a file, uploads it to S3, and creates an SQS message containing the S3 location
pull - Reads in an SQS message, downloads the referenced file, then removes the message from the SQS queue

In push mode, file is renamed using the following convention: <unix time>.<hostname>.log.gz

------------------
Requirements:
-boto, a Python interface for Amazon Web Services (http://code.google.com/p/boto/).
-AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, read in from same-named environment 
variables, from boto.cfg (see boto manual), or from a specified config file (see README).

------------------
Usage: aws_logmover.py [options]

Options:
  -h, --help            show this help message and exit
  -m MODE, --mode=MODE  mode of the task ("push" or "pull")
  -q QUEUE, --queue=QUEUE
                        Amazon SQS queue name (ex: "acme_queue_1")
  -f FILE, --file=FILE  (push) file to be moved (ex: "/var/log/httpd.log.1.gz")
  -b BUCKET, --bucket=BUCKET
                        (push) Amazon S3 bucket name (ex: "acme_bucket_1")
  -o DIR, --output-dir=DIR
                        (pull) output directory for retrieved file (ex:
                        "/tmp/scratch")

Something went wrong with that request. Please try again.