@adammoody adammoody released this Sep 25, 2018 · 30 commits to master since this release

Assets 3

Bug fixes:

  • fix lseek positioning when overwriting an existing file with data from a newer source file when using dsync --contents

md5sum: acbd5b5c15919a67392509614bb7871e

@adammoody adammoody released this Aug 31, 2018 · 30 commits to master since this release

Assets 3

New features:

  • dchmod: added --owner option to change user on files
  • dchmod: fixed bug when using 'a' option in symbolic notation
  • dcmp: added expressions to compare permissions and ACL on files
  • dcp: fix bug that prevented some subdirectories from being created
  • dcp1: original dcp from LANL, may be faster than dcp for directory trees with lots of small files, a similar algorithm to be merged into dcp in future release
  • dfilemaker: updated to create multiple directory levels, files of different sizes with random content, and symlinks
  • dfilemaker1: dfilemaker from v0.7, to be merged into dfilemaker in future release
  • dstripe: added support for Lustre 2.5
  • dsync: new tool to synchronize one directory tree with another (good for backups or completing partial copies)
  • libmfu: added API calls to define new list elements and set their properties
  • libmfu: added function to write file list to text file

New experimental tools (work in progress):

  • dbz2: compress a single file with bz2
  • dfind: filter file lists with find-like tests
  • dgrep: parallel grep
  • dtar: parallel tar (incomplete)

Known bugs:

  • dtar uses libarchive that assumes a single process is writing the tar archive. It likely will generate corrupt tar archives if run with more than one process. This will be fixed in a future release.
  • The binary format for reading and writing filelists to files uses MPI I/O external32 data representation. This has proved to be buggy across MPI implementations. In a future release, mpiFileUtils will be changed to read/write these files with POSIX I/O. If a work around is needed, one may change "external32" to "native" in src/common/mfu_flist_io.c.
  • dsync --contents has a bug when computing its offset during lseek for overwriting an existing file (fixed in v0.8.1)

Updated dependency:

  • Requires update of DTCMP from v1.0.3 to v1.1.0 for new DTCMP_Segmented_scanv/exscanv calls.

md5sum: 1082600e7ac4e6b2c13d91bbec40cffb

@dsikich dsikich released this Jun 23, 2017 · 226 commits to master since this release

Assets 3
  • dbcast: now creates destination directory before broadcasting file
  • dbcast: added --size option to set file segment size
  • dchmod: process umask if user provides no ugoa letter in symbolic notation
  • dcmp: added --sync option to synchronize source and target directory trees
  • dsh: added support for wildcard filters, limit ls output to 100 items by default
  • dsh: added --output and --file options to save modified flist on exit
  • dstripe: now recursively processes files under a directory
  • dstripe: added --report to print file striping info
  • dstripe: added --count and --size options to set stripe parameters
  • dstripe: added --minsize option to only process files above a certain size
  • libmfu: moved copy logic from dcp to new mfu_flist_copy routine
  • libmfu: write flist cache files using multiple stripes, one per MPI rank
  • configure: fixed --enable-experimental builds

Known bugs:

  • using an input list with dcp is broken in v0.7, but this is patched on the main branch

md5sum: c081f7f72c4521dddccdcf9e087c5a2b

@adammoody adammoody released this May 11, 2017 · 348 commits to master since this release

Assets 3

This is an intermediate release along the roadmap for v1.0.

  • dchmod: added dchmod to set access permissions and group
  • added node2 test suite and integrated with travis.ci
  • dcmp: parallelized compare for large data files (now in addition to large directories)
  • dcp: promote dcp2 to dcp, renamed dcp to dcp1
  • renamed common library from libbayer to libmfu (mpiFileUtils)
  • moved significant code to libmfu
  • numerous bugs fixes and other enhancements
Jun 15, 2015
tag second llnl release
Apr 18, 2015
add makedist

@adammoody adammoody released this Aug 2, 2014 · 576 commits to master since this release

Assets 3

add make dist support
May 23, 2014
LLNL release snapshot

@bringhurst bringhurst released this Jan 14, 2014 · 673 commits to master since this release

Assets 2
  • License updates.
  • Vagrant configuration with ansible.
  • Updates to dtest.
  • Updates to easy build scripts.
  • Memory leak fix.

@bringhurst bringhurst released this Dec 13, 2013 · 704 commits to master since this release

Assets 2
  • A basic system for building man pages.
  • Configure time options to enable or disable experimental utilities.
  • Many small documentation fixes.
  • Initial travis-ci configuration.
  • Minor build warning fixes.
  • Initial stubs for nose test suite and dfilemaker.