1.2.0 variable.mask is True but the values are not masked

[agrouaze]

I noticed that in netCDF4 1.2.0 the variable.mask is True but the values are not masked.

print nc.variables['oswLon'][:]

array([[ 9.96920997e+36]], dtype=float32)

<type 'netCDF4._netCDF4.Variable'>

float32 oswLon(oswAzSize, oswRaSize)

units: degree (decimal)

long_name: longitude

unlimited dimensions:

current shape = (1, 1)

filling off

nc.variables['oswLon'][0,0]==netCDF4.default_fillvals['f4']

True

nc.variables['oswLon'].mask

True

[jswhit]

hard to tell what you mean by 'values are not masked'. If you read the data from the file, i.e.

data = nc.variables['oswLon'][:]

then data should be a masked array, and data[0,0] should be masked.

[agrouaze]

Absolutely and this is the reason why I raised this issue, the value is equal to 9.96920997e+36 it should be 'masked'. Does the C libraries (netcdf-bin package) version can have an impact on such things?

[jswhit]

Please submit a script we can run to trigger the problem. I tried it myself and it works as expected, so there must be something different about your dataset. For example, it could be that the fill_value for tht variable is not set to the default.

[agrouaze]

Hi Jeff,
Here is the netcdf file and the script that allow to reproduce the problem.
note that it works perfectly with netCDF4 1.0.4 but with 1.12.0 the
values are no more numpy.ma.masked_constant even with attrbiute mask=True
Cheers

Le 01/10/2015 22:55, Jeff Whitaker a écrit :

Please submit a script we can run to trigger the problem. I tried it
myself and it works as expected, so there must be something different
about your dataset. For example, it could be that the fill_value for
tht variable is not set to the default.

—
Reply to this email directly or view it on GitHub
#468 (comment).

Antoine GROUAZEL
IFREMER LOS/CERSAT
+33 (0)2 29 00 85 55

[jswhit]

The file and script did not come through.

[jswhit]

I still need your test script and data.

[agrouaze]

Hi Jeff,
Is it github that did not allow the attachments?
Anyway you can download them here:
ftp://ftp.ifremer.fr/ifremer/cersat/users/agrouaze/netcdf4_lib/
Cheers

Le 05/10/2015 19:09, Jeff Whitaker a écrit :

I still need your test script and data.

—
Reply to this email directly or view it on GitHub
#468 (comment).

Antoine GROUAZEL
IFREMER LOS/CERSAT
+33 (0)2 29 00 85 55

[jswhit]

It appears that variable filling is disabled in your dataset (when you print the variable, it says 'filling off'). Masked arrays won't be returned if variable filling is off, and the variable does not have the _FillValue attribute explicitly set (see issue #209). This is because when data providers turn off fill mode, it is often because they want to use the full range of the data (including the default fill_value, which is usually set to the largest representable number for the data type).

[agrouaze]

Hi Jeff,
So it is not bug.
I suppose we can close this ticket.
Thanks for your work!
Cheers

On 2015-10-06 16:29, Jeff Whitaker wrote:

It appears that variable filling is disabled in your dataset (when you
print the variable, it says 'filling off'). Masked arrays won't be
returned if variable filling is off, and the variable does not have
the _FillValue attribute explicitly set (see issue #209 [1]). This is
because when data providers turn off fill mode, it is often because
they want to use the full range of the data (including the default
fill_value, which is usually set to the largest representable number
for the data type).

Reply to this email directly or view it on GitHub [2].

Links:

[1] #209
[2]
#468 (comment)

[jswhit]

Thanks for the report, closing now...