Permalink
Commits on Sep 27, 2012
  1. Merge pull request #12 from amorton/duplicate-backup-files

    remove duplicate backup_files() funcs
    JeremyGrosser committed Sep 27, 2012
  2. Merge pull request #11 from eventbrite/issues/10

    Fixes Issue #10
    JeremyGrosser committed Sep 27, 2012
Commits on Sep 26, 2012
  1. Fixes Issue #10

    Appending a last slash to the prefix ensures that namespaces that
    are prefixed with the same string don't get mixed.
    Also PEP8
    thekad committed Sep 26, 2012
Commits on Sep 25, 2012
  1. Merge remote-tracking branch 'librato/master', bump to version 0.6.0

    Conflicts:
    	setup.py
    	tablesnap
    JeremyGrosser committed Sep 25, 2012
  2. Allow UploadHandler class and backup_files() to be passed log param

    I'm importing these methods and using them in a larger construct
    where it's very useful for me to set up my own logging.  I figure
    I may not be the only one.
    Ozzie Sabina committed with JeremyGrosser Sep 24, 2012
  3. Merge pull request #7 from apigee/queue-refactor

    Queue refactor
    JeremyGrosser committed Sep 25, 2012
  4. Merge branch 'queue-refactor' of github.com:apigee/tablesnap into que…

    …ue-refactor
    
    Conflicts:
    	tableslurp
    tnine committed Sep 25, 2012
Commits on Sep 21, 2012
  1. removed unused finally block

    tnine committed Sep 21, 2012
  2. fixed incorrect comparison

    tnine committed Sep 21, 2012
  3. Merge pull request #4 from eventbrite/feature/multipart

    Feature/multipart
    JeremyGrosser committed Sep 21, 2012
Commits on Sep 5, 2012
  1. Fixed wording to be more explicit

    One has to assume the original file uploaded (the very first time) is
    the right and canonical version. If files change somehow it's most
    likely to be a disk corruption issue
    thekad committed Sep 5, 2012
  2. This now requires boto >= 2.2

    There are some functions not available in previous versions of python
    boto, so I am bumping the version
    thekad committed Sep 5, 2012
  3. Actual working version with multipart uploading

    Why I didn't use FileChunkIO? That's a very good question. While
    FileChunkIO is a damn fine idea, is still relatively new and packages
    don't exist for most distributions. Yes, it exists in in pypi but that's
    about it. Also, I'm trying to keep the number of non-stdlib components
    to a minimum.
    
    Currently this will read chunks into memory (by default, 256M at a time,
    but that's in a parameter) and stream them up to S3. IF you are so low
    in usable memory such that 256M is not enough, it will use 1/20th of
    as much free memory as you have.
    
    Another big change is that the MD5 is calculated before uploading and
    stored as key metadata. The reason is that multipart files won't report
    the right MD5 in the ETag (this only for multipart uploads):
    
    https://forums.aws.amazon.com/thread.jspa?messageID=203436
    thekad committed Sep 5, 2012
Commits on Aug 31, 2012
  1. Multipart upload works

    Reading data in 100M chunks ensures that only 100M are in memory and in
    transit at a time. I thought about uploading in parallel but that would
    require threads within threads, which would be more complex to
    understand. Instead, I realized that files are getting uploaded in a
    serialized form anyway, so no need to add complexity
    thekad committed Aug 31, 2012
Commits on Aug 30, 2012
  1. Now you can set the max size and chunk size

    Mostly for testing purposes. I can test with smaller files. Also
    using StringIO to read the file in chunks, if this doesn't really
    work I will need filechunkio, but I really want to avoid using
    external libraries as much as possible
    thekad committed Aug 30, 2012
Commits on Aug 29, 2012
  1. First pass at multipart

    This is most likely not complete yet and will most likely fail, but I
    have to push often :)
    thekad committed Aug 29, 2012
Commits on Aug 24, 2012
  1. Bump version

    mheffner committed Aug 24, 2012
Commits on Aug 17, 2012
  1. Merge pull request #3 from eventbrite/master

    This is the right pull req
    JeremyGrosser committed Aug 17, 2012
Commits on Aug 9, 2012
  1. Bumping version

    thekad committed Aug 9, 2012
Commits on Aug 7, 2012
Commits on Aug 6, 2012
  1. Remove unnecessary variable

    thekad committed Aug 6, 2012
Commits on Aug 5, 2012
  1. Adding script to setup.py

    thekad committed Aug 5, 2012
  2. Adding `tableslurp` command

    Perhaps needs some more testing, but this looks fairly usable already.
    See the --help for documentation on how this one works. I tried to make
    the options as close to the ones in tablesnap as possible with some
    added threading sugar
    thekad committed Aug 5, 2012
Commits on Aug 3, 2012
  1. Merge branch 'librato-0.4.3'

    Conflicts:
    	tablesnap
    thekad committed Aug 3, 2012
Commits on Jul 20, 2012
  1. Merge pull request #1 from librato/bugfix/recursive_upload

    On startup recursively backup files if the recurse option is set.
    mheffner committed Jul 20, 2012
Commits on Feb 9, 2012
  1. Bump version

    mheffner committed Feb 9, 2012