Missing dimension in NetCDF4 file written using netCDF4-0.9

[dopplershift]

From sourish....@gmail.com on February 13, 2010 10:18:52

What steps will reproduce the problem? 1. From the python prompt, run the attached script. Then issue
x = TestNCDF4()
x.WriteFile()

1. After that, 'ncdump test.nc4' does not show the 'time' dimension.
   Instead, it shows 'latitude' twice, and the arrays 'conc' and 'surface
   pressure' show dimensions of latitude x latitude x longitude instead of
   time x latitude x longitude
2. The resulting file can also be viewed using netCDF4 as follows:

from netCDF4 import Dataset
fid = Dataset('test.nc4','r')
fid.dimensions

This will show only two dimensions ('latitude' and 'longitude') and not three. What is the expected output? What do you see instead? I expect to see three dimensions, 'time', 'latitude' and 'longitude' in the
written file. Instead, I only see two. What version of the product are you using? On what operating system? netcdf-4.1
netCDF4-0.9
Ubuntu linux x86_64

Thanks,

-Sourish

Attachment: TestNCDF4.py

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=36

[dopplershift]

From sourish....@gmail.com on February 13, 2010 09:20:40

P.S. - I'm using python 2.6

-Sourish

[dopplershift]

From whitaker.jeffrey@gmail.com on February 13, 2010 14:41:05

Confirmed. This simple script will trigger the bug:

from netCDF4 import Dataset
f = Dataset('test.nc','w',format='NETCDF4_CLASSIC')
f.createDimension('lat', 10)
f.createDimension('lon', 20)
lon = f.createVariable('lon', 'f4', 'lon')
lat = f.createVariable('lat', 'f8', 'lat')
f.close()

Changing the format to NETCDF3_CLASSIC, or reversing the order of creation of the
lon and lat vars makes it go away.

I'm trying to figure out if this is a library bug or a python module bug. Will let you
know when I have more to report.

Thanks for the report.

[dopplershift]

From whitaker.jeffrey@gmail.com on February 13, 2010 14:48:12

Another piece of data. If you build netcdf4-python against netcdf-4.0.1, or netcdf-4.1-
rc1, the bug goes away. It only shows up with netcdf-4.1. Looks like a library bug. I'll
see if I can create a simple c program that triggers it, and then report it to unidata.

[dopplershift]

From whitaker.jeffrey@gmail.com on February 13, 2010 16:06:01

I've confirmed it's a library bug. Sent a simple c program that triggers it to the
netcdfgroup mailing list. Will keep you posted - hopefully a 4.1.1 release will be
forthcoming. In the meantime you can work around it by reversing the order that the
dimensions are created.

[dopplershift]

From sourish....@gmail.com on February 13, 2010 22:49:17

Thanks for the quick response. Two questions:

1. What is the difference between 'NETCDF4_CLASSIC' and 'NETCDF4'? Is it that the
   latter uses HDF5 structures, and the former does not?
2. When you say 'reverse the order', what do you mean? That I should create the
   variables in the same order as the dimensions?

[dopplershift]

From sourish....@gmail.com on February 13, 2010 23:04:58

Confirmed that the bug does not show up with NetCDF-4.1-rc1

[dopplershift]

From whitaker.jeffrey@gmail.com on February 14, 2010 05:55:28

1. NETCDF4_CLASSIC uses the HDF5 storage layer, but the NETCDF3 data model.
2. Instead of messing with the creation order, just stick with 4.1-rc1 till Unidata
   releases a bugfix.

[dopplershift]

From whitaker.jeffrey@gmail.com on February 18, 2010 09:38:21

An update from the netcdf developers:

"The bug itself manifests itself under the following circumstances: if
you define dimensions, then you define a coordinate variable, and write
to it, and then another coordinate variable, and if the coordinate
variables are in a different order than the dimensions, on re-reading
the file, netcdf-4.1 will be confused.

A key part of activating the bug is that you write some of the
coordinate variable data before defining all the coordinate
variables. That is, if you call all the nc_def_var calls for the
coordinate variables before writing any of their data, things will work
fine. You must also be writing the dimensions and the coordinate
variables in different orders to see this bug. If you defined the
coordinate variables in the same order as you defined the dimensions,
writing them before defining all of them would not be a problem.

The way to check for this problem is to do an ncdump -h of the file and
see if the dimension information in the ncdump is correct. In C it can
be detected by closing the file, re-opening it, and checking the
dimension ids of the variables."

[dopplershift]

From whitaker.jeffrey@gmail.com on February 27, 2010 07:18:13

This bug should now be fixed in the hdf5 and netcdf libs. Please test with hdf-1.8.4-
patch1 and netcdf-4.1.1-rc1. Thanks for reporting this.

-Jeff

[dopplershift]

From whitaker.jeffrey@gmail.com on February 25, 2014 18:04:09

Status: Fixed