Summary of HDF5 parallel features

kif edited this page Nov 10, 2014 · 7 revisions

This table sums up what is and is not possible with concurrent accesses on HDF5 files using the h5py library.

  • P: possible? yes/no/MPI required/...
  • F: fast? yes/no/serial/multicore/...
  • LC: limitation causes? h5py/GIL/libhdf5/...

Access mode:

  • R: all processes/threads open the file in read-only mode.
  • W: all processes/threads open the file in write-only mode.
  • RW: some processes/threads read, others write.
<td>Yes</td>
<td>Yes</td>
<td>H5py serialization on Posix FS</td>

<td>-</td>
<td>-</td>
<td>-</td>
<td>-</td>
<td>-</td>
<td>-</td>

<td>Yes</td>
<td>Yes</td>
<td>H5Py serialization on Posix FS</td>
<td>Yes</td>
<td>Yes</td>
<td>H5Py serialization on Posix FS</td>

<td>Yes</td>
<td>Yes</td>
<td>H5Py serialization on Posix FS</td>
<td>R</td>

<td>Yes</td>
<td>Yes</td>
<td>Filesystem</td>

<td>-</td>
<td>-</td>
<td>-</td>
<td>-</td>
<td>-</td>
<td>-</td>

<td>Yes</td>
<td>Yes</td>
<td>Multiple writers possible when using MPI and MPI-IO on a parallel FS</td>
<td>Yes</td>
<td>Yes</td>
<td>limited by filesystem</td>

<td>Only one</td>
<td>Untested</td>
<td>One writer in SWRM mode (no structure change), needs Posix FS, HDF5 1.10</td>
opened in access mode reading writing
P? F? LC? P? F? LC?
one process, multiple threads R
W
RW
multiple processes
W
RW
Clone this wiki locally
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.