Parallel netCDF I/O #187

Open
jhamman opened this Issue Jan 10, 2015 · 0 comments

Comments

Projects
None yet
2 participants
@jhamman
Member

jhamman commented Jan 10, 2015

I found my notes on the comment/question I was going to ask a few weeks back on #23. I'm going to put them here so I don't loose them again and we can close the issue if we decide not to use parallel netCDF I/O.

Parallel access to netCDF file routines are available from two sources:

netCDF4

netCDF4 is is supported in C, C++, Fortran, Python, Java, etc. Parallel I/O support is likely limited to the C and Fortran languages.

Summary from netCDF4 Parallel I/O documentation:

NetCDF-4 provides access to HDF5 parallel I/O features for netCDF-4/HDF5 files.

NetCDF classic and 64-bit offset format may not be opened or created for use with parallel I/O. (They may be opened and created, but parallel I/O is not available.)

A few functions have been added to the netCDF C API to handle parallel I/O.

PnetCDF

Parallel netCDF (PnetCDF) is supported in C, Fortran 77, Fortran 90, and C++.

A summary from the PnetCDF Github page:

Parallel netCDF (PnetCDF) is a library providing high-performance parallel I/O
while still maintaining file-format compatibility with Unidata's NetCDF,
specifically the CDF and CDF-2 formats. Although NetCDF supports parallel I/O
starting from version 4, the files must be in HDF5 format. PnetCDF is currently
the only choice for carrying out parallel I/O on files that are in classic
formats (CDF-1 and 2).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment