Handle utf-8 netcdf.h from netcdf 4.3.1.1 in python3

[opoplawski]

For the Fedora netcdf4-python package, we are building with python3 support. With netcdf 4.3.1.1 we are getting:

+ /usr/bin/python3 setup.py build
HDF5_DIR environment variable not set, checking some standard locations ..
checking /builddir ...
checking /usr/local ...
checking /sw ...
checking /opt ...
checking /opt/local ...
checking /usr ...
HDF5 found in /usr
NETCDF4_DIR environment variable not set, checking standard locations.. 
checking /builddir ...
checking /usr/local ...
checking /sw ...
checking /opt ...
checking /opt/local ...
checking /usr ...
Traceback (most recent call last):
  File "setup.py", line 223, in <module>
    isnetcdf4 = check_ifnetcdf4(os.path.join(direc, 'include'))
  File "setup.py", line 37, in check_ifnetcdf4
    for line in f:
  File "/usr/lib64/python3.3/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 6698: ordinal not in range(128)

This is because netcdf.h is now a utf-8 file. Unfortunately I have not yet found a way to handle this that works with both python2 and python3.

[jswhit]

I'm looking at the version of netcdf.h in github master and I don't see any utf-8 characters in there. Building also works for me with both python2.7 and python3.3 (and all tests pass). I wonder where those non-ascii characters came from? Perhaps your netcdf.h is corrupted somehow?

[opoplawski]

The file is utf-8:

$ find -name netcdf.h | xargs file
./netcdf-c/include/netcdf.h:         C++ source, UTF-8 Unicode text
./netcdf-c-4.3.1.1/include/netcdf.h: C++ source, UTF-8 Unicode text

the line is here:

 *  underlying data‥

My paste ere changed the character it seems - the original has a '.' in the middle of the line. I'll report that to netcdf.

[jswhit]

I see it now. I still don't understand why you're getting that error. python2.7 is happy to read that line in as a string without raising a UnicodeDecodeError for me (on MacOS and Ubuntu). Maybe it has something to do with the Fedora environment.

It should be fixed in netcdf-c though.

[jswhit]

The header is now fixed in netcdf-c master.