S3 can!
JavaScript
Latest commit 47844ad Jan 2, 2017 @tmcw tmcw committed with rclark Move package to @mapbox namespace. (#21)
Permalink
Failed to load latest commit information.
bin Update usage, readme Mar 14, 2016
cloudformation hello, travis Aug 19, 2015
lib fix list stream object mode (#20) Dec 18, 2016
test fix list stream object mode (#20) Dec 18, 2016
.eslintrc Add eslint Feb 9, 2016
.travis.yml hello, travis Aug 19, 2015
api.md Doc cleanup Mar 15, 2016
index.js Doc cleanup Mar 15, 2016
package.json Move package to @mapbox namespace. (#21) Jan 2, 2017
readme.md Update usage, readme Mar 14, 2016

readme.md

Build Status

s3scan

Provides native Node.js streams for S3 operations such as

  • listing all the keys in a bucket/prefix
  • retrieving all the objects under a given bucket/prefix
  • deleting all the children of a given bucket/prefix

Also provides CLI tools:

s3keys: prints a line-delimited list of keys to stdout

$ s3keys s3://my-bucket/some-prefix

s3purge: deletes all objects beneath the provided prefix

$ s3purge s3://my-bucket/all-finished-with-these

s3scan: GET and print a prefix-worth of objects to stdout

$ s3scan s3://my-bucket/some-prefix
$ s3scan s3://my-bucket/some-prefix --gunzip
$ s3scan s3://my-bucket/some-prefix --gunzip --concurrency=1

Running tests

You can run tests against your own bucket/prefix by specifying them as environment variables:

TestBucket=my-bucket TestPrefix=my-prefix npm test

Otherwise these values default to buckets owned by Mapbox which require appropriate authentication.