S3 parallel uploader
Switch branches/tags
Nothing to show
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.


What is bigbucket.py

bigbucket.py is a small python script that allows you to upload full folders to Amazon S3 quickly. To do so it uploads multiple files simultaneously by opening multiple connections to Amazon S3.


Well, if you need to upload thousands of files to S3 takes a looong time with most command clients. With this command tool this just work faster.

If you need similar functionality on FTP, lftp is an incredible tool that can do that.

How to use it

  • Modify the bigbucket.py file and add your {AWS KEY} and {AWS SECRET}

  • Run it via command line like, it looks like this:

      $ python bigbucket.py {dirname} {bucket} {threads} {test} {prefix}


  • dirname - the directory with files you wish to upload
  • bucket - the name of you S3 bucket


  • threads - default = 32
  • test - if True the files will not be pushed to S3
  • prefix - for storing a directory as a subdirectory in an existing bucket


The main developer of the tool is Andrew W Hill (@andrewxhill)