Skip to content

Commit

Permalink
enh: add interface to view and set hdf5 options (#153)
Browse files Browse the repository at this point in the history
* add interface to view and set hdf5 chunking and compression options
* add compact storage option

the supported options are:

chunking/enabled = "true" | "false"
  should chunking be used

chunking/threshold = (int)
  a threshold number of bytes in dataset before using chunking

chunking/chunk_size = (int)
  the size of chunks (in bytes) to be used

chunking/compression/method = "none" | "gzip"
  compression method

chunking/compression/level = (int)
  compression level option for gzip

compact_storage/enabled = "true" | "false"
  should compact storage be used

compact_storage/threshold = (int)
  a threshold max number of bytes in dataset to use compact storage

These options appear in relay::about() and can be set using:

io::hdf5_set_options(n);

Which is included in the conduit_relay_hdf5 header.

* add link of hdf5 for using with make example 

to solve issues with static build testing

* hdf5 group creation property tweaks

* add helper to check if hdf5 path exists

* enable compact by default, simplify chunking logic
  • Loading branch information
cyrush committed Apr 26, 2017
1 parent 9980c5e commit 9ef9871
Show file tree
Hide file tree
Showing 8 changed files with 539 additions and 30 deletions.
4 changes: 2 additions & 2 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -130,8 +130,8 @@ script:
- ./example
# using with make example
- cd ${TRAVIS_BUILD_DIR}/src/examples/using-with-make
- env CXX=${CONDUIT_CXX} CONDUIT_DIR=${TRAVIS_BUILD_DIR}/travis-debug-install make
- env LD_LIBRARY_PATH=${TRAVIS_BUILD_DIR}/travis-debug-install/lib/ ./example
- env CXX=${CONDUIT_CXX} CONDUIT_DIR=${TRAVIS_BUILD_DIR}/travis-debug-install HDF5_DIR=${TRAVIS_HOME}/miniconda/lib make
- env LD_LIBRARY_PATH=${TRAVIS_BUILD_DIR}/travis-debug-install/lib/:${TRAVIS_HOME}/miniconda/lib ./example

after_success:
- test ${ENABLE_COVERAGE} = "ON" && coveralls --gcov /usr/bin/gcov-4.8 --include src/libs/conduit --include src/libs/blueprint --gcov-options '\-lp' --root $TRAVIS_BUILD_DIR --build-root $TRAVIS_BUILD_DIR/travis-debug-build;
Expand Down
2 changes: 1 addition & 1 deletion src/examples/using-with-make/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@


INC_FLAGS=-I$(CONDUIT_DIR)/include/conduit
LINK_FLAGS=-L$(CONDUIT_DIR)/lib/ -lconduit -lconduit_relay -lconduit_blueprint
LINK_FLAGS=-L$(CONDUIT_DIR)/lib/ -lconduit -lconduit_relay -lconduit_blueprint -L$(HDF5_DIR) -lhdf5

main:
$(CXX) $(INC_FLAGS) example.cpp $(LINK_FLAGS) -o example
Expand Down
6 changes: 6 additions & 0 deletions src/libs/relay/conduit_relay.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,10 @@

#include "conduit_relay.hpp"

#ifdef CONDUIT_RELAY_IO_HDF5_ENABLED
#include "conduit_relay_hdf5.hpp"
#endif

//-----------------------------------------------------------------------------
// standard lib includes
//-----------------------------------------------------------------------------
Expand Down Expand Up @@ -103,6 +107,8 @@ about(Node &n)
#ifdef CONDUIT_RELAY_IO_HDF5_ENABLED
// straight hdf5
io_protos["hdf5"] = "enabled";

io::hdf5_options(n["io/options/hdf5"]);
#else
// straight hdf5
io_protos["hdf5"] = "disabled";
Expand Down

0 comments on commit 9ef9871

Please sign in to comment.