Permalink
Browse files

updates to man pages

  • Loading branch information...
dsikich committed Jun 15, 2017
1 parent 2f8e6d4 commit 049f304b193dc3b1e8684d2d4e1bcb507c08692e
View
@@ -1,4 +1,4 @@
.\" Automatically generated by Pandoc 1.19.1
.\" Automatically generated by Pandoc 1.19.2
.\"
.TH "DBCAST" "1" "" "" ""
.hy
@@ -7,40 +7,52 @@
dbcast \- distributed broadcast
.SH SYNOPSIS
.PP
\f[B]dbcast CHUNK_SIZE SRC_FILE DEST_FILE\f[]
\f[B]dbcast [OPTION] SRC DEST\f[]
.SH DESCRIPTION
.PP
Parallel MPI application to recursively broadcast a file from a global
file system to node\-local storage.
Parallel MPI application to recursively broadcast a single file from a
global file system to node\-local storage, like ramdisk or an SSD.
.PP
dbcast helps one copy a source file to node\-local storage on each
compute node.
.PP
The file is logically sliced at chunk boundaries as specified by
CHUNK_SIZE and collectively copied from a global file system to
The file is logically sliced into chunks and collectively copied from a
global file system to node\-local storage.
The source file SRC must be readable by all MPI processes.
The destination file DEST should be the full path of the file in
node\-local storage.
The CHUNK_SIZE parameter should be specified in bytes, and one may use
units like "MB" or "GB".
It is recommended to use the stripe size of a file if this is known.
The path to the source file should be given in SRC_FILE.
The source file should be in a path readable by all MPI processes.
The destination path is given in DEST_FILE.
The destination file should be a path to a node\-local storage location,
like ramdisk or an SSD.
.PP
In its current implementation, the tool requires at least two MPI
processes per compute node, and all compute nodes should have the same
If needed, parent directories for the destination file will be created
as part of the broadcast.
.PP
In the current implementation, dbcast requires at least two MPI
processes per compute node, and all compute nodes must run an equal
number of MPI processes.
.SH OPTIONS
.TP
.B \-s, \-\-size SIZE
The chunk size in bytes used to segment files during the broadcast.
Units like "MB" and "GB" should be immediately follow the number without
spaces (ex.
2MB).
The default size is 1MB.
It is recommended to use the stripe size of a file if this is known.
.RS
.RE
.TP
.B \-h, \-\-help
Print the command usage, and the list of options available.
.RS
.RE
.SH EXAMPLES
.IP "1." 3
To broadcast a file to /ssd on each node slicing at 1MB chunks:
To broadcast a file to /ssd on each node:
.PP
mpirun \-np 128 dbcast 1MB /global/path/to/filenane /ssd/filename
mpirun \-np 128 dbcast /global/path/to/filenane /ssd/filename
.IP "2." 3
Same thing, but slicing at 10MB chunks:
.PP
mpirun \-np 128 dbcast \-s 10MB /global/path/to/filenane /ssd/filename
.IP "3." 3
To read the current striping parameters of a file on Lustre:
.PP
lfs getstripe /path/to/file
lfs getstripe /global/path/to/filename
.SH SEE ALSO
.PP
\f[C]dbcast\f[] (1).
View
@@ -1,4 +1,4 @@
.\" Automatically generated by Pandoc 1.19.1
.\" Automatically generated by Pandoc 1.19.2
.\"
.TH "DCHMOD" "1" "" "" ""
.hy
@@ -10,7 +10,7 @@ dchmod \- distributed tool to set permissions and group
\f[B]dchmod [OPTION] PATH...\f[]
.SH DESCRIPTION
.PP
Parallel MPI application to recurseively change permissions and/or group
Parallel MPI application to recursively change permissions and/or group
from a top level directory.
.PP
dchmod provides functionality similar to \f[I]chmod(1)\f[] and
@@ -31,19 +31,23 @@ Change group to specified GROUP name.
.RE
.TP
.B \-m, \-\-mode MODE
The mode that you want the file or directory to be.
The mode to apply to each item.
MODE may be octal or symbolic syntax similar to chmod(1).
In symbolic notation, "ugoa" are supported as are "rwxX".
As with chmod, if no leading letter "ugoa" is provided, mode bits are
combined with umask to determine the actual mode.
.RS
.RE
.TP
.B \-\-exclude REGEX
Do not modify items whose full path matches REGEX (regexec).
Do not modify items whose full path matches REGEX, processed by
regexec(3).
.RS
.RE
.TP
.B \-\-match REGEX
Only modify items whose full path matches REGEX (regexec).
Only modify items whose full path matches REGEX, processed by
regexec(3).
.RS
.RE
.TP
@@ -55,10 +59,9 @@ full path.
.TP
.B \-v, \-\-verbose
Run in verbose mode.
Prints a list of statistics/timing data for the command.
How many files walked, how many levels there are in the tree, and how
many files the command operated on.
This option also prints the files/sec for each of those.
Prints a list of statistics including the number of files walked, the
number of levels there are in the directory tree, and the number of
files the command operated on, and the files/sec rate for each of those.
.RS
.RE
.TP
@@ -72,13 +75,12 @@ Use octal mode to change permissions:
.PP
mpirun \-np 128 dchmod \-\-mode 755 /directory
.IP "2." 3
Use symbolic mode to change permissions (you can list modes for user,
group, etc if you separate them with a comma):
Set group and mode in a single command using symbolic mode:
.PP
mpirun \-np 129 dchmod \-\-group mygroup \-\-mode u+r,g+rw /directory
mpirun \-np 128 dchmod \-\-group mygroup \-\-mode u+r,g+rw /directory
.IP "3." 3
Change permissions to u+rw on all files/directories EXCEPT those whose
name match regex:
Change permissions to u+rw on all items EXCEPT those whose name match
regex:
.PP
mpirun \-np 128 dchmod \-\-name \-\-exclude \[aq]afilename\[aq] \-\-mode
u+rw /directory
View
@@ -1,4 +1,4 @@
.\" Automatically generated by Pandoc 1.19.1
.\" Automatically generated by Pandoc 1.19.2
.\"
.TH "DCMP" "1" "" "" ""
.hy
@@ -7,28 +7,22 @@
dcmp \- distributed compare
.SH SYNOPSIS
.PP
\f[B]dcmp [OPTION] SOURCE DEST \f[]
\f[B]dcmp [OPTION] SRC DEST \f[]
.SH DESCRIPTION
.PP
Parallel MPI application to compare two files or to recurseively compare
files with same prefix path within two different directories.
This tool is similar to \f[I]cmp(1)\f[].
It reports how many files in two different directories are the same (or
different).
Parallel MPI application to compare two files or to recursively compare
files with same relative paths within two different directories.
.PP
dcmp provides functionality similar to a recursive \f[I]cmp(1)\f[].
It reports how many files in two different directories are the same (or
different).
.SH OPTIONS
.TP
.B \-b, \-\-base
Do a base comparison.
.RS
.RE
.TP
.B \-o, \-\-output <FIELD0=STATE0@FIELD1=STATE1,FIELD2=STATE2:FILE>
write output fields and states to a file.
.RS
.RE
.TP
.B \-v, \-\-verbose
Run in verbose mode.
Prints a list of statistics/timing data for the command.
View
@@ -1,4 +1,4 @@
.\" Automatically generated by Pandoc 1.19.1
.\" Automatically generated by Pandoc 1.19.2
.\"
.TH "DCP" "1" "" "" ""
.hy
@@ -7,7 +7,7 @@
dcp \- distributed copy
.SH SYNOPSIS
.PP
\f[B]dcp [OPTION] SOURCE DEST\f[]
\f[B]dcp [OPTION] SRC DEST\f[]
.SH DESCRIPTION
.PP
Parallel MPI application to recursively copy files and directories.
@@ -51,11 +51,11 @@ Print a brief message listing the \f[I]dcp(1)\f[] options and usage.
.SH RESTRICTIONS
.PP
If a long\-running copy is interrupted, one should delete the partial
copy and rerun dcp from the beginning.
copy and run dcp again from the beginning.
One may use drm to quickly remove a partial copy of a large directory
tree.
.PP
To ensure the copy was successful, one should run dcmp after dcp
To ensure the copy is successful, one should run dcmp after dcp
completes to verify the copy, especially if dcp was not run with the \-s
option.
.SH EXAMPLES
@@ -76,9 +76,7 @@ mpirun \-np 128 dcp \-p /source/dir1/ /dest/dir2
Using the \-S option for sparse files does not work yet at LLNL.
If you try to use it then dcp will default to a normal copy.
.PP
The \-g option is still being developed.
.PP
The maximum supported filename length for any file transfered is
The maximum supported file name length for any file transferred is
approximately 4068 characters.
This may be less than the number of characters that your operating
system supports.
@@ -1,4 +1,4 @@
.\" Automatically generated by Pandoc 1.19.1
.\" Automatically generated by Pandoc 1.19.2
.\"
.TH "DFILEMAKER" "1" "" "" ""
.hy
@@ -19,7 +19,7 @@ they actually exist? Unless there is another version of the code
somewhere?\f[]
.TP
.B \-d, \-\-depth=\f[I]min\f[]\-\f[I]max\f[]
Specify the depth of the filesystem tree to generate.
Specify the depth of the file system tree to generate.
The depth will be selected at random within the bounds of min and max.
The default depth is set to 10 min, 20 max.
.RS
@@ -60,7 +60,7 @@ The default file size is set from 1MB to 5MB.
.RE
.TP
.B \-w, \-\-width=\f[I]min\f[]\-\f[I]max\f[]
Specify the width of the filesystem tree to generate.
Specify the width of the file system tree to generate.
The width will be selected at random within the bounds of min and max.
The width of the tree is determined by counting directories.
The default width is set to 10 min, 20 max.
@@ -88,4 +88,4 @@ Print version information and exit.
\f[C]dwalk\f[] (1).
.PP
The mpiFileUtils source code and all documentation may be downloaded
from <http://fileutils.io>
from <https://github.com/hpc/mpifileutils>
View
@@ -1,4 +1,4 @@
.\" Automatically generated by Pandoc 1.19.1
.\" Automatically generated by Pandoc 1.19.2
.\"
.TH "DFIND" "1" "" "" ""
.hy
View
@@ -1,4 +1,4 @@
.\" Automatically generated by Pandoc 1.19.1
.\" Automatically generated by Pandoc 1.19.2
.\"
.TH "DGREP" "1" "" "" ""
.hy
View
@@ -1,4 +1,4 @@
.\" Automatically generated by Pandoc 1.19.1
.\" Automatically generated by Pandoc 1.19.2
.\"
.TH "DPARALLEL" "1" "" "" ""
.hy
@@ -35,4 +35,4 @@ Print version information and exit.
\f[C]dwalk\f[] (1).
.PP
The mpiFileUtils source code and all documentation may be downloaded
from <http://fileutils.io>
from <https://github.com/hpc/mpifileutils>
View
@@ -1,4 +1,4 @@
.\" Automatically generated by Pandoc 1.19.1
.\" Automatically generated by Pandoc 1.19.2
.\"
.TH "DRM" "1" "" "" ""
.hy
@@ -13,12 +13,8 @@ drm \- distributed remove
Parallel MPI application to recursively delete a directory and its
contents.
.PP
drm is a tool for removing files recurseivly in parallel.
drm is similar to \f[I]rm(1)\f[], but unlike \f[I]rm(1)\f[] there is no
need to pass a recursive option as files will be removed from the top
level directory to the bottom of the file tree by default.
.PP
Be careful: it behaves like \f[I]rm \-rf\f[], but it is much faster.
drm is a tool for removing files recursively in parallel.
Be careful: drm behaves like \f[I]rm \-rf\f[], but it is much faster.
.SH OPTIONS
.TP
.B \-i, \-\-input FILE
@@ -36,14 +32,15 @@ Walk file system without stat.
.IP
.nf
\f[C]
Do\ not\ remove\ items\ whose\ full\ path\ matches\ REGEX\ (regexec).
Do\ not\ remove\ items\ whose\ full\ path\ matches\ REGEX,\ processed\ by\ regexec(3).
\f[]
.fi
.RS
.RE
.TP
.B \-\-match REGEX
Only remove items whose full path matches REGEX (regexec).
Only remove items whose full path matches REGEX, processed by
regexec(3).
.RS
.RE
.TP
@@ -54,9 +51,10 @@ full path.
.RE
.TP
.B \-d, \-\-dryrun
Print out a list of files that \f[B]would\f[] be deleted.
Print a list of files that \f[B]would\f[] be deleted without deleting
them.
This is useful to check list of items satisfying \-\-exclude or
\-\-match options without deleting them.
\-\-match options before actually deleting anything.
.RS
.RE
.TP
@@ -80,7 +78,7 @@ directory tree:
.PP
mpirun \-np 128 drm \-\-match \[aq].core$\[aq] /dir/to/delete/from
.IP "3." 3
List items that would be deleted, without removing them:
List items that would be deleted without removing them:
.PP
mpirun \-np 128 drm \-\-dryrun \-\-match \[aq].core$\[aq]
/dir/to/delete/from
Oops, something went wrong.

0 comments on commit 049f304

Please sign in to comment.