Accessing fields for a masked structured array fails with ValueError

[gerritholl]

I'm trying to create a mask for a structured array. When I try to access a field from the array or simply represent it, numpy throws a ValueError: field names A not found, as illustrated below.

>>> R = numpy.empty(10, dtype=[("A", "<f2"), ("B", "<f4")])
>>> Rm = numpy.ma.masked_where(R["A"]<0.1, R)
>>> Rm["A"]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/local/gerrit/python3.2-bleed/lib/python3.2/site-packages/numpy/ma/core.py", line 3014, in __getitem__
    dout._mask = _mask[indx]
ValueError: field named A not found.
>>> print(Rm)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/local/gerrit/python3.2-bleed/lib/python3.2/site-packages/numpy/ma/core.py", line 3583, in __str__
    _recursive_printoption(res, m, f)
  File "/local/gerrit/python3.2-bleed/lib/python3.2/site-packages/numpy/ma/core.py", line 2294, in _recursive_printoption
    (curdata, curmask) = (result[name], mask[name])
ValueError: field named A not found.
>>> print(numpy.version.version)
1.8.0.dev-b8bfcd0
>>> print(numpy.version.git_revision)
b8bfcd02a2f246a9c23675e1650c3d316d733306

I tested it with the stable version 1.6.2 and the bleeding-edge version obtained directly from git.
It also fails in earlier stable versions (tested with 1.6.2).

Note: if I instead create directly a masked array with numpy.ma.empty, I get a different error:

>>> R2 = numpy.ma.empty(10, dtype=[("A", "<f2"), ("B", "<f4")])
>>> Rm2 = numpy.ma.masked_where(R2["A"]<0.1, R2)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/local/gerrit/python3.2-bleed/lib/python3.2/site-packages/numpy/ma/core.py", line 1810, in masked_where
    cond = mask_or(cond, a._mask)
  File "/local/gerrit/python3.2-bleed/lib/python3.2/site-packages/numpy/ma/core.py", line 1627, in mask_or
    raise ValueError("Incompatible dtypes '%s'<>'%s'" % (dtype1, dtype2))
ValueError: Incompatible dtypes 'bool'<>'[('A', '?'), ('B', '?')]

[gerritholl]

I don't know if this is the same issue, but with newer numpy versions, the error message has changed:

>>> import sys, os
>>> print(numpy.version.version, numpy.version.git_revision, sys.version, os.uname())
1.9.0.dev-a0cf183 a0cf18394d5ce33514fdc37093bd2f65ad4b0dde 3.4.0 (default, Apr  3 2014, 09:15:09) 
[GCC 4.8.1] posix.uname_result(sysname='Linux', nodename='ostrovnoy', release='3.11.0-19-generic', version='#33-Ubuntu SMP Tue Mar 11 18:48:34 UTC 2014', machine='x86_64')
>>> A = numpy.ma.masked_all(shape=5, dtype=[("A", "f4", 3)])
>>> b = A[0]
>>> b["A"]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/gerrit/venv/python-3.4-bleed/lib/python3.4/site-packages/numpy/ma/core.py", line 5619, in __getitem__
    if m is not nomask and m[indx]:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

[gerritholl]

Still present in version 1.10.0.dev+00ee332 but with a yet different error message:

In [131]: R = numpy.empty(10, dtype=[("A", "<f2"), ("B", "<f4")])

In [132]: Rm = numpy.ma.masked_where(R["A"]<0.1, R)

In [133]: Rm["A"]
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-133-7f4318176b14> in <module>()
----> 1 Rm["A"]

/export/data/home/gholl/venv/bleeding/lib/python3.4/site-packages/numpy/ma/core.py in __getitem__(self, indx)
   3071             # Update the mask if needed
   3072             if _mask is not nomask:
-> 3073                 dout._mask = _mask[indx]
   3074                 dout._sharedmask = True
   3075 #               Note: Don't try to check for m.any(), that'll take too long...

IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices

[gerritholl]

I started to fix this, but as I realised the problem is caused by Rm.mask.dtype being unstructured (type bool), although Rm.dtype is structured. Is this supposed to be possible? I asked at Stack Overflow: http://stackoverflow.com/q/28182408/974555