Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
S3 parallel uploader
Python
branch: master

Fetching latest commit…

Cannot retrieve the latest commit at this time

Failed to load latest commit information.
README.md
bigbucket.py

README.md

What is bigbucket.py

bigbucket.py is a small python script that allows you to upload full folders to Amazon S3 quickly. To do so it uploads multiple files simultaneously by opening multiple connections to Amazon S3.

Why?

Well, if you need to upload thousands of files to S3 takes a looong time with most command clients. With this command tool this just work faster.

If you need similar functionality on FTP, lftp is an incredible tool that can do that.

How to use it

  • Modify the bigbucket.py file and add your {AWS KEY} and {AWS SECRET}

  • Run it via command line like, it looks like this:

    $ python bigbucket.py {dirname} {bucket} {threads} {test} {prefix}
    

where:

  • dirname - the directory with files you wish to upload
  • bucket - the name of you S3 bucket

optional:

  • threads - default = 32
  • test - if True the files will not be pushed to S3
  • prefix - for storing a directory as a subdirectory in an existing bucket

Authors:

The main developer of the tool is Andrew W Hill (@andrewxhill)

Something went wrong with that request. Please try again.