corrupted netcdf files being created (not the same problem as before...)

[dopplershift]

From andy...@gmail.com on July 13, 2010 05:36:01

What steps will reproduce the problem? 1. Run the attached script
2. Run ncdump on the outputted file What is the expected output? What do you see instead? netcdf test {
dimensions:
latitude = 10 ;
latitude = 10 ;
time = 1 ;
variables:
double time_bounds(time, latitude) ;
float latitude(latitude) ;
double time(time) ;
data:

time_bounds =
ncdump: NetCDF: Start+count exceeds dimension bound

I would expect to see the dimensions latitude, nv and time, with the variable time_bounds corresponding to dimensions 'nv' and 'time'. What version of the product are you using? On what operating system? I am using Fedora 13, so the package versions are:
netcdf4-python 0.9.1 (built from source)
netcdf 4.1.1 (Fedora package)
hdf5 1.8.4.patch1 (Fedora package) Please provide any additional information below. I attempted to translate the python program into the equivalent C program, as follows:

include <netcdf.h>

int main() {
int dataset_id;
int lat_id, nv_id, time_id;
int lat_var_id, time_bounds_var_id, time_var_id;

nc_create("test_c.nc", NC_NETCDF4|NC_CLASSIC_MODEL, &dataset_id);

nc_def_dim(dataset_id, "latitude", 10, &lat_id);
nc_def_dim(dataset_id, "nv", 2, &nv_id);
nc_def_dim(dataset_id, "time", 1, &time_id);

const int time_bounds_dims[2] = {time_id, nv_id};
nc_def_var(dataset_id, "time_bounds", NC_DOUBLE, 2, time_bounds_dims, &time_bounds_var_id);
nc_def_var(dataset_id, "latitude", NC_FLOAT, 1, &lat_id, &lat_var_id);
nc_def_var(dataset_id, "time", NC_DOUBLE, 1, &time_id, &time_var_id);

nc_sync(dataset_id);
nc_close(dataset_id);

}
This creates a valid netcdf when run on the same system with the same libraries.

Attachment: minimal_test_case.py

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=58

[dopplershift]

From whitaker.jeffrey@gmail.com on July 13, 2010 06:04:26

This appears to be the bug

* Fix netCDF-4 bug caused when doing enddef/ redef and then defining
   coordinate variable out of order. 

that was recently fixed in netcdf-4.1.2-beta1. I bet if you put enddef/reddefs around the nc_def_dims and nc_def_vars in your c code it would create the same corrupted file.

I verified that your script works if the python module is linked against netcdf-4.1.2-beta1. Unfortunately, there are other problems with the beta1 release, so I don't recommend you upgrade. The only workaround I can suggest now is re-ordering the dimension creation (if you put the definition of the "nv" dimension before "time", it works).

-Jeff

[dopplershift]

From andy...@gmail.com on July 13, 2010 08:04:55

I have worked around as you suggested (updated the order of creation so that coordinate variables are created in the same order as their corresponding dimensions), and this creates working files.

Thanks a lot for your help
Andy

[dopplershift]

From whitaker.jeffrey@gmail.com on July 13, 2010 12:10:41

I have reported the bug to unidata, so I will update the ticket when there is a new 4.1.2 beta release that solves this problem (without causing others, as beta1 does).

-Jeff

[dopplershift]

From whitaker.jeffrey@gmail.com on August 20, 2010 09:22:30

This bug appears to be fixed using netcdf from svn and hdf5-1.8.5-snap4, so when netcdf 4.1.2-beta2 comes out this issue should be solved.

[dopplershift]

From MikeGran...@googlemail.com on August 25, 2013 13:51:59

Confirmed working on Fedora 17 (netcdf-4.2-3.fc17.x86_64, hdf5-1.8.8-8.fc17.x86_64, netCDF4 0.9.9 & 1.0.5). Should be alright to close this now.

[dopplershift]

From whitaker.jeffrey@gmail.com on February 25, 2014 18:04:09

Status: Fixed