Permalink
Commits on Jan 11, 2010
  1. Merge branch 'cygwin'

    * cygwin:
      Assorted cleanups to Luke's cygwin fixes.
      Makefile: work with cygwin on different windows versions.
      .gitignore sanity.
      Makefile:  On Windows, executable files must end with .exe.
      client.py:  Windows files don't support ':', so rename cachedir.
      index.py:  os.rename() fails on Windows if dstfile already exists.
      Don't try to rename tmpfiles into existing open files.
      helpers.py:  Cygwin doesn't support `hostname -f`, use `hostname`.
      cmd-index.py:  Retry os.open without O_LARGEFILE if not supported.
      Makefile:  Build on Windows under Cygwin.
    committed Jan 11, 2010
  2. Assorted cleanups to Luke's cygwin fixes.

    There were a few things that weren't quite done how I would have done them,
    so I changed the implementation.  Should still work in cygwin, though.
    
    The only actual functional changes are:
     - index.Reader.close() now actually sets m=None rather than just closing it
     - removed the "if rename fails, then unlink first" logic, which is
       seemingly not needed after all.
     - rather than special-casing cygwin to use "hostname" instead of "hostname
       -f", it turns out python has a socket.getfqdn() that does what we want.
    committed Jan 11, 2010
  3. Makefile: work with cygwin on different windows versions.

    Just check the CYGWIN part; don't depend on the fact that it's NT 5.1.  (Of
    course, uname isn't supposed to report such things by default anyway... but
    that's cygwin for you.)
    committed Jan 11, 2010
Commits on Jan 10, 2010
  1. .gitignore sanity.

    lkosewsk committed Jan 10, 2010
  2. client.py: Windows files don't support ':', so rename cachedir.

    Cachedir was previously $host:$dir, and is now $host-$dir.
    lkosewsk committed Jan 10, 2010
  3. index.py: os.rename() fails on Windows if dstfile already exists.

    Hence, we perform an os.unlink on the dstfile if os.rename() receives
    an OSError exception, and try again.
    lkosewsk committed Jan 10, 2010
  4. Don't try to rename tmpfiles into existing open files.

    Linux and friends have no problem with this, but Windows doesn't allow
    this without some effort, which we can avoid by... not needing to write
    to an already-open file.
    
    Give index.Reader a 'close' method which identifies and closes any open
    mmaped files, and make cmd-index.py use this before trying to close a
    index.Writer instance (which renames a tmpfile into the same file the
    Reader has mmaped).
    lkosewsk committed Jan 10, 2010
  5. cmd-index.py: Retry os.open without O_LARGEFILE if not supported.

    Python under Cygwin doesn't have os.O_LARGEFILE, so if we receive an
    'AttributeError' exception trying to open something, just remove
    O_LARGEFILE and try again.
    lkosewsk committed Jan 10, 2010
  6. Makefile: Build on Windows under Cygwin.

    - Python modules have to end with .dll instead .so to load into Python
      via 'import'.
    - GCC under Windows builds all programs with -fPIC, and doesn't accept
      this command-line option.
    - libpython2.5.dll is found in /usr/bin under Cygwin (wtf?), so we need
      to add this to the LDFLAGS path.
    - 'make clean' should remove .dll files too.
    lkosewsk committed Jan 10, 2010
  7. Oops, 'bup save /' produced an invalid tree.

    Add a bunch of assertions to make sure that never happens.
    committed Jan 10, 2010
  8. This adds the long-awaited indexfile feature, so you no longer have t…

    …o feed
    
    your backups through tar.
    
    Okay, 'bup save' is still a bit weak... but it could be much worse.
    
    Merge branch 'indexfile'
    
    * indexfile:
      Minor fix for python 2.4.4 compatibility.
      cmd-save: completely reimplement using the indexfile.
      Moved some reusable index-handling code from cmd-index.py to index.py.
      A bunch of wvtests for the 'bup index' command.
      Start using wvtest.sh for shell-based tests in test-sh.
      cmd-index: default indexfile path is ~/.bup/bupindex, not $PWD/index
      cmd-index: skip merging the index if nothing was written to the new one.
      cmd-index: only update if -u is given; print only given file/dirnames.
      cmd-index: correct reporting of deleted vs. added vs. modified status.
      Generalize the multi-index-walking code.
      cmd-index: indexfiles should start with a well-known header.
      cmd-index: eliminate redundant paths from index update command.
      cmd-index: some handy options.
      index: add --xdev (--one-file-system) option.
      Fix some bugs with indexing '/'
      cmd-index: basic index reader/writer/merger.
    committed Jan 10, 2010
  9. cmd-save: completely reimplement using the indexfile.

    'bup save' no longer walks the filesystem: instead it walks the indexfile
    (which is much faster) and doesn't bother opening any files that haven't had
    an attribute change, since it can just reuse their sha1 from before.  That
    makes it *much* faster in the common case.
    committed Jan 10, 2010
  10. Start using wvtest.sh for shell-based tests in test-sh.

    This makes the output a little prettier... at least in the common case where
    it passes :)
    committed Jan 10, 2010
  11. cmd-index: only update if -u is given; print only given file/dirnames.

    cmd-index now does two things:
     - it updates the index with the given names if -u is given
     - it prints the index if -p, -s, or -m are given.
    
    In both cases, if filenames are given, it operates (recursively) on the
    given filenames or directories.  If no filenames are given, -u fails (we
    don't want to default to /; it's too slow) but -p/s/m just prints the whole
    index.
    committed Jan 10, 2010
  12. cmd-index: correct reporting of deleted vs. added vs. modified status.

    A file with an all-zero sha1 is considered Added instead of Modified, since
    it has obviously *never* had a valid sha1.  (A modified file has an old
    sha1, but IX_HASHVALID isn't set.)
    
    We also now don't remove old files from the index - for now - so that we can
    report old files with a D status.  This might perhaps be useful eventually.
    
    Furthermore, we had a but where reindexing a particular filename would
    "sometimes" cause siblings of that file to be marked as deleted.  The
    sibling entries should never be updated, because we didn't check them and
    thus have no idea of their new status.  This bug was mostly caused by the
    silly way we current pass dirnames and filenames around...
    committed Jan 10, 2010
Commits on Jan 9, 2010
  1. Generalize the multi-index-walking code.

    Now you can walk through multiple indexes correctly from anywhere, avoiding
    the need for merging a huge index just to update a few files.
    committed Jan 9, 2010
Commits on Jan 8, 2010
  1. cmd-index: eliminate redundant paths from index update command.

    If someone asks to update "/etc" and "/etc/passwd", the latter is redundant
    because it's included in the first.  Don't bother updating the file twice
    (and thus causing two index merges, etc).
    
    Ideally we would only do one merge for *any* number of updates (etc /etc and
    /var).  This should be possible as long as we sort the entries correctly
    (/var/ and then /etc/), since a single sequential indexfile could just have
    one appended to the other.  But we don't do that yet.
    committed Jan 7, 2010
  2. cmd-index: some handy options.

    New options:
    	--modified: print only files that aren't up to date
    	--status: prefix printouts with status chars
    	--fake-valid: mark all entries as up to date
    	--indexfile: override the default index filename
    committed Jan 7, 2010
  3. index: add --xdev (--one-file-system) option.

    For not traversing across filesystem boundaries.
    committed Jan 7, 2010
Commits on Jan 7, 2010
  1. On MacOS X, "wc -l" returns extra whitespace.

    ie. "    0" instead of "0".
    
    The easiest workaround is to compare as a number instead of as a string.
    This seems to work correctly on both MacOS and Linux.
    committed Jan 7, 2010
  2. More compile options for MacOS X.

    Based on a patch from Dave Coombs.  I changed it to auto-detect the OS
    platform, so I might have broken it, however.
    committed Jan 7, 2010
Commits on Jan 6, 2010
  1. splitting to a remote server would cause "already busy" errors.

    Specifically:
    	client.ClientError: already busy with command 'receive-objects'
    
    That's because recent changes removed the call to onclose() from
    PackWriter_Remote.  Now it's back, plus I added an extra unit test to reveal
    the problem.
    committed Jan 6, 2010
  2. client: enhance the PATH when searching for the 'bup' binary.

    Automatically adds the *local* $PWD to the *remote* $PATH before trying to
    run 'bup server'.  That way, if you build the source in exactly the same
    folder on two machines - or if those two machines are actually the same
    machine and you're just doing a test against localhost - it'll work.
    
    I hereby curse both "sh -c <command>" and "ssh hostname -- <command>" for
    not allowing a sensible way to just set argv[] without doing any stupid
    quoting.  Nasty.
    committed Jan 6, 2010