-
Notifications
You must be signed in to change notification settings - Fork 262
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
NetCDF: HDF error when creating a lot of variables and attributes #2251
Comments
Further investigation reveals that the location returns the error is at Lines 1412 to 1413 in cd0f169
In issue #1822, @brtnfld mentioned that "HDF5 does not test any of the HL APIs in a parallel setting". Given that, I wonder if netcdf plans to resolve the bug by taking @brtnfld's suggestion. |
I wonder if this PR #2161 |
Yes. Thanks. |
* NetCDF-4 uses the dimension scale feature which is part of HDF5 high-level APIs, but HDF5 high-level APIs are not well tested for parallel I/O. See Unidata/netcdf-c#2251 and Unidata/netcdf-c#1822 * NetCDF PR #2161 adds a new flag NC_NODIMSCALE_ATTACH to allow users to disable dimension scale, which resolves the problem for e3sm-io. See Unidata/netcdf-c#2161 * NetCDF team indicates PR #2161 will appear in version 4.9.0.
* NetCDF-4 uses the dimension scale feature which is part of HDF5 high-level APIs, but HDF5 high-level APIs are not well tested for parallel I/O. See Unidata/netcdf-c#2251 and Unidata/netcdf-c#1822 * NetCDF PR #2161 adds a new flag NC_NODIMSCALE_ATTACH to allow users to disable dimension scale, which resolves the problem for e3sm-io. See Unidata/netcdf-c#2161 * NetCDF team indicates PR #2161 will appear in version 4.9.0.
* NetCDF-4 uses the dimension scale feature which is part of HDF5 high-level APIs, but HDF5 high-level APIs are not well tested for parallel I/O. See Unidata/netcdf-c#2251 and Unidata/netcdf-c#1822 * NetCDF PR #2161 adds a new flag NC_NODIMSCALE_ATTACH to allow users to disable dimension scale, which resolves the problem for e3sm-io. See Unidata/netcdf-c#2161 * NetCDF team indicates PR #2161 will appear in version 4.9.0.
* NetCDF-4 uses the dimension scale feature which is part of HDF5 high-level APIs, but HDF5 high-level APIs are not well tested for parallel I/O. See Unidata/netcdf-c#2251 and Unidata/netcdf-c#1822 * NetCDF PR #2161 adds a new flag NC_NODIMSCALE_ATTACH to allow users to disable dimension scale, which resolves the problem for e3sm-io. See Unidata/netcdf-c#2161 * NetCDF team indicates PR #2161 will appear in version 4.9.0.
* NetCDF-4 uses the dimension scale feature which is part of HDF5 high-level APIs, but HDF5 high-level APIs are not well tested for parallel I/O. See Unidata/netcdf-c#2251 and Unidata/netcdf-c#1822 * NetCDF PR #2161 adds a new flag NC_NODIMSCALE_ATTACH to allow users to disable dimension scale, which resolves the problem for e3sm-io. See Unidata/netcdf-c#2161 * NetCDF team indicates PR #2161 will appear in version 4.9.0.
NetCDF 4.8.1
HDF5 1.13.0
MPICH 3.4.3
gcc 8.5.0
I encountered "NetCDF: HDF error" when running a parallel program that creates
an HDF5-based NetCDF4 file. The test program that can reproduce the error is
available in icase_def.c
The test program follows the E3SM I/O pattern by creating the a large number of
variables and attributes:
27 global attributes
21 dimensions
560 variables
and each variable has a few attributes.
The test program does not call any nc_put_var* APIs.
When using one MPI process, the test program ran fine.
But when running 4 MPI processes, it printed the following errors and hung
at nc_enddef.
The text was updated successfully, but these errors were encountered: