Permalink
Commits on Nov 9, 2010
  1. Add make export-docs/push-docs/import-docs targets.

    export-docs: update local 'man' and 'html' branches with pregenerated doc
    files.
    
    push-docs: push those to origin/man and origin/html.
    
    import-docs: extract the documentation from origin/man and origin/html into
    the local tree so it can be installed.
    
    Signed-off-by: Avery Pennarun <apenwarr@gmail.com>
    committed Nov 9, 2010
  2. Revert (most of) "--remote parameter requires a colon"

    This reverts (most of) commit c135a58.
    
    Requiring a colon seems to be too fascist; it makes people think that you
    can't use local repositories anymore, which wasn't true: you could just
    refer to them as ":/path/to/repo".  But that's just too weird and
    non-obvious.  It already resulted in a query on the mailing list, the
    avoidance of which is why we added this patch in the first place.  So let's
    take it back out.
    
    I kept some minor clarifications and unit test improvements, however.
    
    Signed-off-by: Avery Pennarun <apenwarr@gmail.com>
    committed Nov 9, 2010
  3. Add a coding style document.

    The document is largely inspired by the one in Scott Chacon's "HACKING"
    file [1] in his 'agitmemnon-server' repository on GitHub with some
    precision on the docstring style that was adopted for bup.
    
    http://github.com/schacon/agitmemnon-server/blob/master/HACKING
    
    Signed-off-by: Gabriel Filion <lelutin@gmail.com>
    lelutin committed with Oct 11, 2010
  4. Revert new-style classes

    Some classes were changed to "new-style" Python classes in c7a0f06.
    
    Following a discussion on the mailing list about the relevance of such a
    change, it was noted that the features that new-style classes bring are
    not used in bup, and considering their slightly higher cost in
    instantiating them and accessing their attributes, it is decided that we
    don't change to using them.
    
    Revert the changed clases back to old-style classes so that all code is
    consistent.
    
    Signed-off-by: Gabriel Filion <lelutin@gmail.com>
    lelutin committed with Oct 11, 2010
  5. If not directory root, prepend list with ".."

    Add a link to the file list to traverse up a directory if we
    are not already at the top
    
    Signed-off-by: David Roda <davidcroda@gmail.com>
    davidcroda committed with Sep 23, 2010
  6. Use relative width for wrapper. Don't stretch table

    Switched wrapper from 960px to 90%
    Removed width from table and columns to allow the
    browser to properly auto align them
    
    Signed-off-by: David Roda <davidcroda@gmail.com>
    davidcroda committed with Sep 23, 2010
Commits on Oct 18, 2010
  1. DESIGN: 'bup restore' exists now.

    Reported by Dieter Plaetinck.
    
    Signed-off-by: Avery Pennarun <apenwarr@gmail.com>
    committed Oct 18, 2010
Commits on Oct 16, 2010
  1. cmd/save: if file.read() returns an error, don't abort.

    Apparently some mis-implemented Linux filesystems (selinuxfs) have regular
    files that can be opened for read, but return EINVAL when you try to read
    them.  We would throw a fatal exception in that case (since we're not
    supposed to have read errors ever, and thus that implies something happened
    that we didn't think of) but I guess we'd better make this into a non-fatal
    error.  It still makes the exit code nonzero so you can see that something
    didn't work, though.
    
    Reported by Zoran Zaric.
    
    Signed-off-by: Avery Pennarun <apenwarr@gmail.com>
    committed Oct 16, 2010
Commits on Oct 4, 2010
  1. Merge remote branch 'origin/master'

    * origin/master:
      cmd/web: stream large files asynchronously.
      cmd/save: oops, missing a mangle_name() call.
    committed Oct 4, 2010
  2. cmd/web: stream large files asynchronously.

    We had a nice chunkyreader() loop for writing files, but unfortunately,
    Tornado captured the full content of those files before writing them to the
    client.  Oops.
    
    Change things around so we don't end up buffering some multiple of the
    ENTIRE FILE in memory.
    
    Signed-off-by: Avery Pennarun <apenwarr@gmail.com>
    committed Oct 4, 2010
  3. cmd/save: oops, missing a mangle_name() call.

    Directories with names ending in '.bup' - including ~/.bup, sigh - didn't
    get the .bupl suffix added, thus making their sizes not calculate correctly.
    
    Signed-off-by: Avery Pennarun <apenwarr@gmail.com>
    committed Oct 4, 2010
Commits on Sep 22, 2010
  1. cmd/split: print a progress counter.

    We don't know how many bytes we're going to split in total, but we can at
    least print the total number of bytes we've seen so far.
    
    Also fix cmd/random to *not* print progress messages by default, since my
    test situation is
    	bup random 100M | bup split -b
    and they scribble over each other when they both print progress output.  bup
    random now gets a '-v' option.
    
    Signed-off-by: Avery Pennarun <apenwarr@gmail.com>
    committed Sep 22, 2010
  2. git.py: support the old git pack .idx version 1 format.

    It's produced by git-fast-import, and I happen to want to read some of
    those.
    
    Signed-off-by: Avery Pennarun <apenwarr@gmail.com>
    committed Sep 22, 2010
  3. git.py: more careful handling of .idx version codes.

    Now it prints a useful error if it sees a version 1 pack (no header) or some
    newer as-yet-undefined version.
    
    Signed-off-by: Avery Pennarun <apenwarr@gmail.com>
    committed Sep 22, 2010
  4. cmd/split: add a --git-ids option.

    This lets you provide a list of git object ids on stdin instead of the raw
    content.  bup-split then uses a CatPipe to retrieve the objects from git and
    hashsplit them.  You could use this as a helper for converting a git repo
    that contains a bunch of large files into one that uses bup-style hashsplit
    files.
    
    Signed-off-by: Avery Pennarun <apenwarr@gmail.com>
    committed Sep 22, 2010
  5. cmd/split: add a new --keep-boundaries option.

    If you provide multiple input files on the command line, sometimes you want
    to merge them togther into a single file before re-chunking them (the
    default).  But sometimes you want all the files to be treated separately for
    chunking purposes, ie. when you know that some of the files will never
    change so there's never any point in merging it with previous/subsequent
    files.
    
    Signed-off-by: Avery Pennarun <apenwarr@gmail.com>
    committed Sep 22, 2010
  6. Merge branch 'dr/web'

    * drweb:
      Add simple styling to bup web.
      If we are showing hidden files, continue to do so.
      Enable static resources. Move css to external file.
    committed Sep 22, 2010
  7. Merge branch 'maint'

    * maint:
      Fixup ctime/mtime that are outside a 32-bit range.
    committed Sep 22, 2010
  8. Add simple styling to bup web.

    This adds a wrapper set to 960px to the bup web layout. It also
    sets widths on the table and columns.
    
    I added a wrapper div to list-directory.html but I think that must
    have snuck in with a previous commit.  I am not sure how to fix that
    so I will leave it for now.  Sorry!
    
    Adds a wrapper div to the html template.
    
    Signed-off-by: David Roda <davidcroda@gmail.com>
    davidcroda committed with Sep 14, 2010
  9. If we are showing hidden files, continue to do so.

    This appends ?hidden=1 to all url's outputted to the template
    if we are currently showing hidden files.  I added a variable
    url_append which is appended to the urls outside of the escaping
    
    Signed-off-by: David Roda <davidcroda@gmail.com>
    davidcroda committed with Sep 22, 2010
  10. Enable static resources. Move css to external file.

    Add to the settings variable in web-cmd.py to set
    /web/static to be servced as static resources.  This is for
    css, javascript, and images.
    
    Move the current styles from the head to static/css/styles.css.
    Remove a few unnecessary styles and change the tab stop
    to 4 spaces to match the rest of the code.
    
    Add to Makefile to copy new directory structure.
    
    Signed-off-by: David Roda <davidcroda@gmail.com>
    davidcroda committed with Sep 14, 2010
Commits on Sep 9, 2010
  1. Fixup ctime/mtime that are outside a 32-bit range.

    This avoids a DeprecationWarning on python 2.6.  That warning probably
    should have been an error instead.
    
    Reported by David Roda.
    
    Signed-off-by: Avery Pennarun <apenwarr@gmail.com>
    committed Sep 9, 2010
Commits on Sep 8, 2010
  1. Adds a date option to save, split and bup.git.PackWriter._new_commit()

    Signed-off-by: Zoran Zaric <zz@zoranzaric.de>
    zoranzaric committed with Sep 8, 2010
  2. --remote parameter requires a colon

    This patch checks for the presence of a colon if the --remote option
    is used in bup save, bup split, bup join, and bup init.  Even though
    specifying *only* a pathname without a hostname: is perfectly valid,
    it's confusing to allow users to do so, because if they specify
    "-r hostname" it will be treated as a path and thus give them a
    confusing error message. Requiring a colon will avoid this.
    
    It adds a few test cases to demonstrate that the code
    works properly.
    
    It also wraps the remote connection in a try except to prevent
    a traceback if there is an error (so far I have only seen this
    happen with an invalid bup dir parameter)
    
    And I added the netbeans project folder to gitignore
    
    Signed-off-by: David Roda <davidcroda@gmail.com>
    davidcroda committed with Sep 8, 2010
  3. cmd/restore: embarrassingly slow implementation of 'bup restore'

    Well, that was easy, since vfs.py already existed and is doing most of the
    hard work.  Only 103 lines including all the log message handling and
    whatnot.
    
    Only one catch: the restoring code is definitely not optimized.  Among other
    things (like the probably-excessive-for-our-restoring-needs layering in
    vfs.py), we're still calling into 'git cat-file --stdin' to retrieve our
    objects.  This involves lots and lots of context switches, plus it can't use
    midx files for its lookups.  The result is that restoring takes much more
    CPU time and memory than it really should.  But oh well, we have to start
    somewhere.
    
    Signed-off-by: Avery Pennarun <apenwarr@gmail.com>
    committed Sep 8, 2010
  4. cmd/save: always print a progress() message after a log() message.

    An earlier commit (634df2f) made 'bup save'
    update the progress line much less frequently.  Unfortunately, if you used
    -v or -vv, this would mean that there was *no* progress bar for a short time
    after every log() message (directory or filename).  That made the progress
    bar flicker annoyingly.
    
    To fix it, make sure we reset the progress bar timer after every filename we
    print with log().  It's subtle, but it makes a very visible difference.
    
    Signed-off-by: Avery Pennarun <apenwarr@gmail.com>
    committed Sep 8, 2010
  5. options.py: get the real tty width for word wrapping purposes.

    Previously we just assumed it was 70 chars, which was safe enough, but not
    as elegant as actually reading the real value and adjusting the word wrap of
    the usage string accordingly.
    
    Signed-off-by: Avery Pennarun <apenwarr@gmail.com>
    committed Sep 8, 2010
  6. options.py: remove extra newlines in usage string.

    If the first line after the "--" was a comment (started with whitespace),
    then we'd end up printing a double newline instead of a single one after the
    synopsis string.
    
    It would also look weird if we had a multi-line comment; the lines would be
    separated by blank lines.
    
    'bup damage -?' encountered this problem.
    
    Signed-off-by: Avery Pennarun <apenwarr@gmail.com>
    committed Sep 8, 2010
  7. options.py: handle optspecs that include inline square brackets.

    We recently made it so if the last thing on an options line was [defval],
    then the value in brackets became the default for that option.  However, we
    inadvertently matched *any* bracketed value on that line, not just the one
    at the end of the line, which basically prevents us from using square
    brackets anywhere on the line.  That's no fun.
    
    Signed-off-by: Avery Pennarun <apenwarr@gmail.com>
    committed Sep 8, 2010
  8. options.py: better support for explicit no-* options.

    If a declared option name starts with no-xxx, then the 'xxx' option starts
    off set to True by default, so that no-xxx is False by default, so that
    passing --no-xxx as an option will have the desired effect of setting
    --no-xxx=True (and thus --xxx=False).
    
    Previously, trying to list a --no-xxx option in the argument list would
    trigger an assertion failure.
    
    Signed-off-by: Avery Pennarun <apenwarr@gmail.com>
    committed Sep 8, 2010
Commits on Sep 7, 2010
  1. Introduce BUP_DEBUG, --debug, and tone down the log messages a lot.

    There's a new global bup option, --debug (-D) that increments BUP_DEBUG.  If
    BUP_DEBUG >=1, debug1() prints; if >= 2, debug2() prints.
    
    We change a bunch of formerly-always-printing log() messages to debug1 or
    debug2, so now a typical bup session should be a lot less noisy.
    
    This affects midx in particular, which was *way* too noisy now that 'bup
    save' and 'bup server' were running it automatically every now and then.
    
    Signed-off-by: Avery Pennarun <apenwarr@gmail.com>
    committed Sep 7, 2010
  2. client.py,git.py: run 'bup midx -a' automatically sometimes.

    Now that 'bup midx -a' is smarter, we should run it automatically after
    creating a new index file.  This should remove the need for running it by
    hand.
    
    Thus, we also remove 'bup midx' from the lists of commonly-used subcommands.
    (While we're here, let's take out 'split' and 'join' too; you should be
    using 'index' and 'save' most of the time.)
    
    Signed-off-by: Avery Pennarun <apenwarr@gmail.com>
    committed Sep 7, 2010
Commits on Sep 6, 2010
  1. Rename 'bup rbackup' to 'bup on'

    'rbackup' was a dumb name but I couldn't think of anything better at the
    time.  This works nicely in a grammatical sort of way:
    
       bup on myserver save -n myserver-backup /etc
    
    Now that we've settled on a name, also add some documentation for the
    command.
    
    Signed-off-by: Avery Pennarun <apenwarr@gmail.com>
    committed Sep 6, 2010
  2. rot13 the t/testfile* sample data files.

    They were generated by catting bunches of bup source code together, which,
    as it turns out, makes 'git grep' super annoying.  Let's rot13 them so
    grepping doesn't do anything interesting but the other characteristics are
    the same.
    
    Signed-off-by: Avery Pennarun <apenwarr@gmail.com>
    committed Sep 6, 2010
  3. cmd/midx: --auto mode can combine existing midx files now.

    Previously, --auto would *only* create a midx from not-already-midxed .idx
    files.  This wasn't optimal since you'd eventually end up with a tonne of
    .midx files, which is just as bad as a tonne of .idx files.
    
    Now we'll try to maintain a maximum number of midx files using a
    highwater/lowwater mark.  That means the number of active midx files should
    now stay between 2 and 5, and you can run 'bup midx -a' as often as you
    want.
    
    'bup midx -f' will still make sure everything is in a single .midx file,
    which is an efficient thing to run every now and then.
    
    'bup midx -af' is the same, but uses existing midx files rather than forcing
    bup to start from only .idx files.  Theoretically this should always be
    faster than, and never be worse than, 'bup midx -f'.
    
    Bonus: 'bup midx -a' now works when there's a limited number of file
    descriptors.  The previous fix only worked properly with 'bup midx -f'.
    (This was rarely a problem since 'bup midx -a' would only ever touch the
    last few .idx files, so it didn't need many file descriptors.)
    
    Signed-off-by: Avery Pennarun <apenwarr@gmail.com>
    committed Sep 2, 2010