Permalink
Commits on Oct 6, 2014
  1. Add some Fedora instructions to the README

    David Brown committed Oct 6, 2014
  2. Ignore "test-pool" directory

    It is useful to have a 'test-pool' directory.  However, ocamlbuild
    likes to descend directories, even symlinks, to look for hygeine
    issues.  With a large pool, this can add quite a delay to the initial
    build time.
    
    Add a line to the _tags file to tell it to not traverse this directory
    at all.
    David Brown committed Oct 6, 2014
  3. Remove dependent projects

    Now that all of our dependent projects are available in opam, remove
    the source here and reference them properly as packages.
    David Brown committed Oct 6, 2014
Commits on Oct 5, 2014
  1. Eliminate warnings in 4.02.0

    Unfortunately, this won't compile with 4.01.0, but the changes are
    minor.
    committed with David Brown Oct 5, 2014
Commits on Mar 16, 2014
  1. Move sql pool into a dir

    Rather than a file and some dirs at the top level, move the database
    into a directory itself.  The resulting format is
    
       poolname/                       - The directory with the pool
       poolname/data.db                - The SQL database
       poolname/blobs/xx/xxx           - Expanded blobs
       poolname/seen/cache-xxx.sqlite  - The "seen" databases.
    David Brown committed Mar 16, 2014
Commits on Mar 15, 2014
  1. Use Pool_util in manager

    Instead of trying to guess whether a pool can be opened, just try.
    David Brown committed Mar 15, 2014
  2. Implement SQL pool

    The SQL pool is an alternate storage format for the backup pools.
    Rather than having to maintain our own index and file formats, store
    the bulk of the information in an sqlite3 database.  Blobs that take
    more than 100k are written to individual files, as per advice from the
    sqlite3 docs.
    
    The pool type is auto-detected upon open, and there are two create
    commands, the original for the older file-based pool, and create-sql
    which will create a new SQL pool.
    
    The seendb is still kept in separate files, now under poolname.seen
    along side the database.
    David Brown committed Mar 15, 2014
  3. Don't warn on 44 and 45

    Ocaml 4.1 generates warnings 44 and 45 when symbols are shadowed.
    This can potentially be useful, but Batteries works by shadowing a
    large number of symbols from the prelude, so this only generates a lot
    of noise.
    David Brown committed Mar 15, 2014
Commits on Aug 13, 2013
  1. Fix backup index on index recovery

    Add entries to the "metadata/backups.txt" file while performing index
    recovery.  This is necessary to truly be able to restore a backup from
    just the data files.
    
    To support this, instead of just appending entries to the
    "metadata/backups.txt" file, read the existing one, do a set add, and
    write the entries out.  This will avoid duplicates that otherwise
    could be created when doing the recovery.
    David Brown committed Aug 13, 2013
Commits on Aug 12, 2013
  1. Remove stray debug files in tmp

    David Brown committed with David Brown Aug 12, 2013
  2. Clarify the newfile logic

    Invert the logic to avoid needing so many negations.  Also, increase
    the number of nodes written so that the test cases can ensure the
    right number of files are created.
    David Brown committed Aug 12, 2013
  3. Implement 'newfile' support

    This config file option tells odump to start with fresh pool files for
    each invocation.  This is useful for certain mirroring situations,
    where files should not be appended to once they have been written.
    
    Currently, this has to be set manually in the pool's
    metadata/props.txt file with a line such as:
    
    	newfile=true
    David Brown committed Aug 12, 2013
  4. Update README for opam

    It is easy to build under opam, give explicity instructions for this.
    David Brown committed Aug 12, 2013
Commits on Feb 5, 2013
  1. Fix timestamp in logs

    The time in the logs was using cpu-time not system time, causing the
    timestamps to generally be near 1970.
    David Brown committed Feb 5, 2013
Commits on Feb 2, 2013
  1. Backup 'xattrs'

    Check each file during the backup, and save any extended attributes
    associated with it.  Restore does not pay attention to these, yet,
    however.
    David Brown committed Feb 2, 2013
  2. Move lvm vg down to per-fs node

    Instead of using the host's 'vol' attribute to determine the volume
    group to use, create an optional attribute under each 'fs', called
    'vg' which specifies the volume group for that particular filesystem.
    This allows a given host backup to span more than a single volume
    group.
    David Brown committed Feb 2, 2013
Commits on Jan 26, 2013
  1. Allow the 'mirror' to be optional

    Only run the rsync commands if the 'mirror' attribute is specified for
    a given host.  This allows configurations that do not have the need
    for the mirror.
    David Brown committed Jan 26, 2013
  2. Implement a backup manager

    Inspired by the Scala version, this performs all of the steps of a
    backup, including snapshots, integrity checks, rsyncs, and the backup
    itself to be done in a single command invocation of odump.  The
    particulars are described in the config file, which gives the hosts
    and the volumes that can be backed up.
    
    The code handles several different filesystem types (with and without
    lvm), and tries to cleanup after itself if something goes wrong.
    David Brown committed Jan 26, 2013
  3. Use the new ksprintf2 calls for user formatting

    Batteries now includes a ksprintf2 call that accepts arguments with
    "%a" that match those available in each package.  This allows things
    such as '"%a" (List.print Int.print) intlist' to be used to print a
    list of integers.
    David Brown committed Jan 26, 2013
  4. Move the backup driver into the backup Module

    This will allow the manager to invoke backups without needing circular
    dependencies in modules.
    David Brown committed Jan 26, 2013
Commits on Jan 25, 2013
  1. Eliminate (most) warnings

    Enable nearly all warnings in the compiler, and eliminate their
    occurrences from the source.  The only one remaining is '4': "Fragile
    pattern matching: matching that will remain complete even if
    additional constructors are added to one of the variant types
    matched."  There are a few types in the program with a bunch of
    constructors that would be tedious to match fully.
    David Brown committed Jan 25, 2013
  2. Port to Batteries 2.0

    Batteries 2.0 makes some minor changes that have a somewhat pervasive
    effect:
      - Use open Batteries instead of open Batteries_uni
      - Std functions moved into pervasives
      - Map and Set no longer contain predefined maps and sets.
      - Map.S.filter becomes Map.S.filterv.
      - Function app (low precedence) is now @@ instead of **>
      - List no longer has labels.
    David Brown committed Jan 23, 2013
Commits on May 13, 2012
  1. Better build outside of godi

    Godi uses the name 'camlzip' instead of 'zip'.  Go back to the
    standard 'zip' name for the package, with comments on how to fix for
    Godi.  Also add some comments to the README describing the packages
    needed to install on Arch linux.
    David Brown committed May 13, 2012
Commits on May 9, 2012
  1. Pool data file verification

    The 'odump verify ...' command will verify specified pool data files
    to make sure the integrity of the hashes is correct.
    
    To implement this, add a new method to the chunk_file class that
    supports checking a chunk, and returning the next offset.  This
    requires some refactoring so that the header information can be
    maintained enough to determine the next location.
    
    The verify loop just walks through the given files and makes sure
    there are chunks that are correct.
    David Brown committed May 9, 2012
  2. Ignore vim's swap files

    David Brown committed May 9, 2012
Commits on Apr 23, 2012
  1. Revert "Remove all dependency on pxp"

    This reverts commit 9260354.
    
    Pxp isn't actually the only cause of bringing in camomile.  Need to
    figure out a way of running camomile without needing the whole of the
    godi tree present.
    David Brown committed Apr 23, 2012
  2. Show path during backup

    As the progress meter updates once a second, show the path currently
    being backed up.  This can help give an idea of the progress of the
    backup.
    David Brown committed Apr 23, 2012
Commits on Apr 17, 2012
  1. Remove all dependency on pxp

    The ocaml pxp library depends on camomile, which requires a bunch of
    runtime files to be present in the install directory.  This makes
    odump more difficult to run without this directory present.  Although
    we eliminate support for reading older-style backups, removing the pxp
    support does eliminate the need to have these files present.
    David Brown committed Apr 17, 2012
Commits on Apr 7, 2012
  1. Convert package to godi

    Adds sources for config-file, and uuidm, which aren't packaged in
    godi, and changes the package name from zip to camlzip.
    
    Makes it a little more of a hastle to install on Debian, but easier on
    other platforms.
    David Brown committed Apr 7, 2012
Commits on Apr 2, 2012
  1. Remove extra .sqlite suffix on cache file

    Only append ".sqlite" to the filename once, avoiding names such as
    "cache-path.sqlite.sqlite".
    David Brown committed Apr 2, 2012
  2. Add a 'version' command

    During build, extract the program version from git (using git
    describe), and make this available as 'odump version'.
    
    This will only work if the tree is actually a git repository (and git
    is installed).
    David Brown committed Apr 2, 2012
  3. Include hostname in cache path

    Since backups can easily be done to network mounts, we need to include
    something to distinguish cache files between different hosts that
    share the same base path.  Use the hostname returned by
    gethostbyname() as part of the name.
    
    Backups should use 'odump show-cache' to determine the new pathnames,
    otherwise, the cache will have to be rebuilt by reading every file in
    the backup source.
    David Brown committed Apr 2, 2012
Commits on Mar 28, 2012
  1. Rename the pool types to pool.mli

    By declaring only an .mli file, the compile will flag an error if this
    signature contains any definitions that would require a module body.
    David Brown committed Mar 28, 2012
Commits on Mar 27, 2012
  1. Extract mountpoint determination code

    This is a bit more than just "Misc".  This is in preparation for
    making this detect filesystem type and uuid information.
    David Brown committed Mar 27, 2012
  2. Add build support for typerex

    Wraps the compilation in the typerex compiler rules, which allows
    typerex to find cross references and such.
    
    The typerex wrapper slows down the compilation quite a bit, so
    switching to byte-compilation to reduce dependencies could be useful.
    David Brown committed Mar 26, 2012