nc4tonc3 chokes on least_significant_digit attribute

[dopplershift]

From david.huard on April 06, 2009 08:15:20

I just needed an utility to convert netcdf4 to netcdf3 and noticed you
added it a couple of days ago !

I tried it and noticed it got

Traceback (most recent call last):
File "/usr/local/bin/nc4tonc3", line 114, in
nc3tonc4(filename4,filename3,clobber=overwritefile,quiet=quiet)
File "/usr/local/bin/nc4tonc3", line 37, in nc3tonc4
setattr(var,attname,getattr(ncvar,attname))
File "netCDF4.pyx", line 1804, in netCDF4.Variable.setattr
(netCDF4.c:12598)
File "netCDF4.pyx", line 714, in netCDF4._set_att (netCDF4.c:3020)
AttributeError: NetCDF: Not a valid data type or _FillValue type mismatch

So I added those lines to nc4tonc3.py the script see what happened

    for attname in ncvar.ncattrs():
        try:            
            setattr(var,attname,getattr(ncvar,attname))
        except:
            print 'Could not set attribute', attname, ':',

getattr(ncvar, attname)

and I got

huardda@angus:~/forcing/wind/80$ nc4tonc3 geowinds.nc geowinds.nc3
copying global attributes ..
copying dimensions ..
copying variable lat
copying variable uwnd
Could not set attribute least_significant_digit : 6

The uwnd variable is created using

That is, my netcdf variable is generated with

uwnd = root.createVariable('uwnd','f4',('time','y','x',),
zlib=True,least_significant_digit=6)

If you can't guess what the problem is right away, I'll hunt it later on.

Original issue: http://code.google.com/p/netcdf4-python/issues/detail?id=15

[dopplershift]

From whitaker.jeffrey@gmail.com on April 06, 2009 08:22:41

David: I just tried it on a file containing a variable with a
least_significant_digit attribute, and did not see that error. If you post the file
for me, I can investigate further.

-Jeff

[dopplershift]

From david.huard on April 06, 2009 08:59:51

The file is 2Gb... I'll have a go at finding the bug.

[dopplershift]

From david.huard on April 06, 2009 09:44:41

Okay, I'm on a 64 bits machine, so the dtype of the least_siginificant_digit is
int64. In the function _set_att, the code checks that the type is in _supportedtypes.
Since it is, there is no need to convert the int64 to int32. Now, int64 are
supported in netCDF4 but not in netCDF3, right ? So the conversion does not take
place when it should.

One solution is to open ncfile3 using netCDF3.Dataset, instead of netCDF4.Dataset,
but I'll let you judge what's the best approach.

[dopplershift]

From david.huard on April 06, 2009 09:58:52

if array is 64 bit integers, cast to 32 bit integers

# if 64-bit datatype not supported.
if value_arr.dtype.str[1:] == 'i8' and _get_format(grpid)[:7]=='NETCDF3':
    value_arr = value_arr.astype('i4')

Is another, simpler, option.

[dopplershift]

From whitaker.jeffrey@gmail.com on April 06, 2009 12:50:24

Just committed that, can you try it out and let me know?

[dopplershift]

From david.huard on April 06, 2009 13:57:01

Looks good.

[dopplershift]

From whitaker.jeffrey@gmail.com on May 25, 2009 05:58:26

Status: Verified