From 2f5f09e0044bf156ce71baf198d1c69ec6c0f7ae Mon Sep 17 00:00:00 2001 From: Chris Hogan Date: Thu, 28 Apr 2022 13:53:57 -0500 Subject: [PATCH 1/2] Update VFD README --- adapter/vfd/README.md | 120 ++++++++++++++++++++++++++++++------------ 1 file changed, 86 insertions(+), 34 deletions(-) diff --git a/adapter/vfd/README.md b/adapter/vfd/README.md index 408b73601..3481a8348 100644 --- a/adapter/vfd/README.md +++ b/adapter/vfd/README.md @@ -1,50 +1,102 @@ # HDF5 Hermes VFD + ## 1. Description -The HDF5 Hermes VFD is a Virtual File Driver (VFD) for HDF5 that can be used to interface with Hermes) API. The driver is built as a plugin library that is external to HDF5. +The HDF5 Hermes VFD is a [Virtual File +Driver](https://portal.hdfgroup.org/display/HDF5/Virtual+File+Drivers) (VFD) for +HDF5 that can be used to interface with the Hermes API. The driver is built as a +plugin library that is external to HDF5. ## 2. Dependencies -To build the HDF5 Hermes VFD, the following libraries are required: -* [Hermes](https://github.com/HDFGroup/hermes) - A heterogeneous aware, multi-tiered, dynamic, and distributed I/O buffering system that aims to significantly accelerate I/O performance. Make sure to build with HERMES_ENABLE_WRAPPER=ON. -* [HDF5](https://github.com/HDFGroup/hdf5) - HDF5 is built for fast I/O processing and storage. - -## 3. Building - -### CMake -Hermes VFD makes use of the CMake build system and requires an out of source build. -``` bash -cd /path/to/Hermes_VFD -mkdir build -cd build -ccmake .. +The Hermes VFD requires [HDF5](https://github.com/HDFGroup/hdf5) >= `1.13.0`, +which is the version that first introduced dynamically loadable VFDs. + +## 3. Usage +To use the HDF5 Hermes VFD in an HDF5 application, the driver can either be +linked with the application during compilation, or it can be dynamically loaded +via an environment variable. It is more convenient to load the VFD as a dynamic +plugin because it does not require code changes or recompilation. + +### Method 1: Dynamically loaded by environment variable (recommended) + +As of HDF5 `1.13.0` each file in an HDF5 app opened or created with the default +file access property list (FAPL) will use the VFD specified in the `HDF5_DRIVER` +environment variable rather than the default "sec2" (POSIX) driver. To use the +Hermes VFD, simply set + +```sh +HDF5_DRIVER=hermes ``` -Type 'c' to configure until there are no errors, then generate the makefile with 'g'. The default options should suffice for most use cases. In addition, we recommend the following options. +Now we must tell the HDF5 library where to find the Hermes VFD. That is done +with the following environment variable: -``` bash --DCMAKE_INSTALL_PREFIX=/installation/prefix --DHDF5_DIR=/paht/to/hdf5 --DHERMES_DIR=/path/to/hermes +```sh +HDF5_PLUGIN_PATH=/hermes.conf +LD_PRELOAD=/hermes_vfd/libhdf5_hermes_vfd.so +``` + +Heres is a full example of running an HDF5 app with the Hermes VFD: + +```sh +HDF5_DRIVER=hermes \ + HDF5_PLUGIN_PATH=/hermes.conf \ + LD_PRELOAD=/hermes_vfd/libhdf5_hermes_vfd.so \ + ./my_hdf5_app +``` -### Method 2: Dynamically loaded by FAPL -To explicitly load the Hermes VFD inside an HDF5 application, a call to the `H5Pset_driver_by_name(...)` routine should be made to setup Hermes VFD access on a FAPL. This will cause HDF5 to load the VFD as a plugin and set the VFD on the given FAPL. The string "hermes" should be given for the driver_name parameter. A string should be given for the `driver_config` parameter (last parameter in `H5Pset_driver_by_name`), as the driver requires additional parameters to config Hermes. An example string like "false,1024" (comma dilemma) is matching the second and third parameter as in `H5Pset_fapl_hermes`. User also needs to set up the enrivonment variable `HDF5_PLUGIN_PATH`, which points to directory containing the built Hermes VFD library. Test `hermes_driver` is using this method. +### Method 2: Linked into application + +To link the Hermes VFD into an HDF5 application, the application should include +the `H5FDhermes.h` header and should link the installed VFD library, +`libhdf5_hermes_vfd.so`, into the application. Once this has been done, Hermes +VFD access can be set up by calling `H5Pset_fapl_hermes()` on a FAPL within the +HDF5 application. In this case, the `persistence` and `page_size` configuration +options are pass directly to this function: + +```C +herr_t H5Pset_fapl_hermes(hid_t fapl_id, hbool_t persistence, size_t page_size) +``` -### Method 3: Dynamically loaded by environment variable -To implicitly load the Hermes VFD inside an HDF5 application, the HDF5_DRIVER environment variable may be set to the string "hermes". During library initialization, HDF5 will check this environment variable, load the Hermes VFD as a plugin and set the VFD as the default file driver on File Access Property Lists. Therefore, any file access that uses H5P_DEFAULT for its FAPL, or which uses a FAPL that hasn't had a specific VFD set on it, will automatically use the Hermes VFD for file access. User can simply setup HDF5_DRIVER environment variable to "hermes" and HDF5_DRIVER_CONFIG the same as `driver_config` parameter without calling `H5Pset_driver_by_name()`, compared to Method 2 Dynamically loaded by FAPL. Test `hermes_env` is using this method. +The resulting `fapl_id` should then be passed to each file open or creation for +which you wish to use the Hermes VFD. -## 5. More Information -[More about Hermes](https://github.com/HDFGroup/hermes/wiki) +## 4. More Information +* [Hermes VFD performance results](https://github.com/HDFGroup/hermes/wiki/HDF5-Hermes-VFD) -[HDF5 VFD Plugins RFC](https://github.com/HDFGroup/hdf5doc/blob/master/RFCs/HDF5_Library/VFL_DriverPlugins/RFC__A_Plugin_Interface_for_HDF5_Virtual_File_Drivers.pdf) +* [Hermes VFD with hdf5-iotest](https://github.com/HDFGroup/hermes/tree/master/benchmarks/HermesVFD) +* [HDF5 VFD Plugins RFC](https://github.com/HDFGroup/hdf5doc/blob/master/RFCs/HDF5_Library/VFL_DriverPlugins/RFC__A_Plugin_Interface_for_HDF5_Virtual_File_Drivers.pdf) From 5442ed7e60c78c67b8e96d719a02aa1d028b40dd Mon Sep 17 00:00:00 2001 From: Chris Hogan Date: Thu, 28 Apr 2022 14:46:08 -0500 Subject: [PATCH 2/2] Fix typos --- adapter/vfd/README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/adapter/vfd/README.md b/adapter/vfd/README.md index 3481a8348..e0a15b501 100644 --- a/adapter/vfd/README.md +++ b/adapter/vfd/README.md @@ -31,7 +31,7 @@ Now we must tell the HDF5 library where to find the Hermes VFD. That is done with the following environment variable: ```sh -HDF5_PLUGIN_PATH=/lib/hermes_vfd ``` The Hermes VFD has two configuration options. @@ -41,7 +41,7 @@ The Hermes VFD has two configuration options. application terminates. Thus, no files are produced. 2. page size - The Hermes VFD works by mapping HDF5 files into its internal data structures. This happens in pages. The `page size` configuration option - allows the user to specify the size, in bytes of these pages. If your app + allows the user to specify the size, in bytes, of these pages. If your app does lots 2 MiB writes, then it's a good idea to set the page size to 2 MiB. A smaller page size, 1 KiB for example, would convert each 2 MiB write into 2048 1 KiB writes. However, be aware that using pages that are too large @@ -51,7 +51,7 @@ The Hermes VFD has two configuration options. These two configuration options are passed as a space-delimited string through -an evnironment variable: +an environment variable: ```sh # Example of configuring the Hermes VFD with `persistent_mode=true` and