Skip to content

Commit

Permalink
add wraprun.sh; update readme on example programs
Browse files Browse the repository at this point in the history
  • Loading branch information
khou2020 committed Apr 25, 2021
1 parent 40576e0 commit b13778e
Show file tree
Hide file tree
Showing 6 changed files with 119 additions and 11 deletions.
29 changes: 24 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,16 +35,35 @@ This software repository contains source codes implementing an [HDF5](https://ww
+ Configure log-based VOL
+ Shared library is required to enable log-based VOL by environment variables
+ Compile with zlib library to enable metadata compression
+ Run test programs (make check) to check whether the VOl is working properly
+ Example commands are given below.
```
% git clone https://github.com/DataLib-ECP/vol-log-based.git
% cd log_io_vol
% autoreconf -i
% ./configure --prefix=${HOME}/Log_IO_VOL --with-hdf5=${HOME}/HDF5 --enable-shared --enable-zlib
% make -j 4 install
% make -j 4
% make check
% make install
```
The VOL plugin library is now installed under the folder `${HOME}/Log_IO_VOL.`

### Running example programs
* Build the log-based VOL
* Compile the example programs under the example directory
+ Run `make <program name>` to compile an example program
+ Running `make tests` will compile all example programs
* Run the example programs
+ Run `make check` to run all example programs as test programs
* Example commands are given below. This example will install
the HD5 library under the folder `${HOME}/HDF5`.
```
% cd example
% make create_open
% ./create_open
Writing file_name = test.h5 at rank 0
```

### Compile user programs that use this VOL plugin
* Enable log-based VOL programmatically
* Include header file.
Expand Down Expand Up @@ -100,14 +119,14 @@ This software repository contains source codes implementing an [HDF5](https://ww
+ Feature to read datasets in log-based storage layout is under development
+ Utility to repack dataset in log-based storage layout into conventional storage layout is under development

### Developers
* Kai-yuan Hou <kai-yuanhou2020@u.northwestern.edu>
* Wei-keng Liao <wkliao@northwestern.edu>

### References
* [HDF5 VOL application developer manual](https://bitbucket.hdfgroup.org/projects/HDFFV/repos/hdf5doc/browse/RFCs/HDF5/VOL/developer_guide/main.pdf)
* [HDF5 VOL plug-in developer manual](https://bitbucket.hdfgroup.org/projects/HDFFV/repos/hdf5doc/browse/RFCs/HDF5/VOL/user_guide)
* [HDF5 VOL RFC](https://bitbucket.hdfgroup.org/projects/HDFFV/repos/hdf5doc/browse/RFCs/HDF5/VOL/RFC)

### Developers
* Kai-yuan Hou <kai-yuanhou2020@u.northwestern.edu>
* Wei-keng Liao <wkliao@northwestern.edu>

### Project funding supports:
This research was supported by the Exascale Computing Project (17-SC-20-SC), a joint project of the U.S. Department of Energy’s Office of Science and National Nuclear Security Administration, responsible for delivering a capable exascale ecosystem, including software, applications, and hardware technology, to support the nation’s exascale computing imperative.
50 changes: 49 additions & 1 deletion example/Makefile.am
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,56 @@
#
# @configure_input@

C_SUBDIRS = hdf5
# C_SUBDIRS = hdf5

#
# Copyright (C) 2020, Northwestern University and Argonne National Laboratory
# See COPYRIGHT notice in top-level directory.
#
# $Id$
#
# @configure_input@

SUFFIXES = .o .cpp

AM_DEFAULT_SOURCE_EXT = .cpp

AM_CPPFLAGS = -I$(top_srcdir)/src \
-I$(top_srcdir)/test \
-I$(srcdir)
LDADD = $(top_builddir)/src/.libs/libH5VL_log.la

check_PROGRAMS = create_open

EXTRA_DIST = # wrap_runs.sh parallel_run.sh

TESTPROGRAMS = $(check_PROGRAMS)

# autimake 1.11.3 has not yet implemented AM_TESTS_ENVIRONMENT
# For newer versions, we can use AM_TESTS_ENVIRONMENT instead
# AM_TESTS_ENVIRONMENT = export TESTPROGRAMS="$(TESTPROGRAMS)";
# AM_TESTS_ENVIRONMENT += export TESTSEQRUN="$(TESTSEQRUN)";
# AM_TESTS_ENVIRONMENT += export TESTOUTDIR="$(TESTOUTDIR)";
TESTS_ENVIRONMENT = export SED="$(SED)";
TESTS_ENVIRONMENT += export srcdir="$(srcdir)";
TESTS_ENVIRONMENT += export TESTOUTDIR="$(TESTOUTDIR)";
TESTS_ENVIRONMENT += export TESTSEQRUN="$(TESTSEQRUN)";
TESTS_ENVIRONMENT += export TESTMPIRUN="$(TESTMPIRUN)";
TESTS_ENVIRONMENT += export TESTPROGRAMS="$(TESTPROGRAMS)";
TESTS_ENVIRONMENT += export check_PROGRAMS="$(check_PROGRAMS)";

H5_FILES = $(check_PROGRAMS:%=$(TESTOUTDIR)/%.h5)

CLEANFILES = $(H5_FILES) core core.* *.gcda *.gcno *.gcov gmon.out

TEST_EXTENSIONS = .sh
LOG_COMPILER = $(srcdir)/wrap_runs.sh
SH_LOG_COMPILER =

TESTS = $(TESTPROGRAMS)

# build check targets but not invoke
tests-local: all $(check_PROGRAMS)

DIST_SUBDIRS = $(C_SUBDIRS)

Expand Down
15 changes: 15 additions & 0 deletions example/hdf5/wrap_runs.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/bin/sh
#
# Copyright (C) 2003, Northwestern University and Argonne National Laboratory
# See COPYRIGHT notice in top-level directory.
#

# Exit immediately if a command exits with a non-zero status.
set -e

outfile=`basename $1`

# export HDF5_VOL_CONNECTOR="LOG under_vol=0;under_info={}"
# export HDF5_PLUGIN_PATH="../../src/.libs"

${TESTSEQRUN} ./$1 ${TESTOUTDIR}/$outfile.h5
15 changes: 15 additions & 0 deletions example/wrap_runs.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/bin/sh
#
# Copyright (C) 2003, Northwestern University and Argonne National Laboratory
# See COPYRIGHT notice in top-level directory.
#

# Exit immediately if a command exits with a non-zero status.
set -e

outfile=`basename $1`

# export HDF5_VOL_CONNECTOR="LOG under_vol=0;under_info={}"
# export HDF5_PLUGIN_PATH="../../src/.libs"

${TESTSEQRUN} ./$1 ${TESTOUTDIR}/$outfile.h5
6 changes: 1 addition & 5 deletions test/basic/Makefile.am
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,7 @@ check_PROGRAMS = file \
group \
attr \
dset \
dwrite \
dread \
memsel
# noclose
# test
dwrite

EXTRA_DIST = # wrap_runs.sh parallel_run.sh

Expand Down
15 changes: 15 additions & 0 deletions test/basic/wrap_runs.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#!/bin/sh
#
# Copyright (C) 2003, Northwestern University and Argonne National Laboratory
# See COPYRIGHT notice in top-level directory.
#

# Exit immediately if a command exits with a non-zero status.
set -e

outfile=`basename $1`

# export HDF5_VOL_CONNECTOR="LOG under_vol=0;under_info={}"
# export HDF5_PLUGIN_PATH="../../src/.libs"

${TESTSEQRUN} ./$1 ${TESTOUTDIR}/$outfile.h5

0 comments on commit b13778e

Please sign in to comment.