Unexpected behaviour on comparing object array with np.nan elements

[chbrandt]

The scenario I'm dealing with is an array of dtype object where this array has some np.nan entries (where original data were missing) and strings where data are valid. I got stuck when trying to get a mask from such array.

Consider the example:

>>> a = np.array([np.nan,'ok'],dtype=np.object)
>>> a
array([nan, 'ok'], dtype=object)

Using np.isnan doesn't work because:

TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

Then I went to basic (IEEE754) rules and compared a with itself (element-wise) expecting I would get False for the "NaN" entry and True for the other one... but nop:

>>> np.equal(a,a)
array([ True,  True], dtype=bool)

And that is my issue; this behaviour is completely unexpected to me.

If I do the elements comparison by my self, it works:

>>> a[0] == np.nan
False
>>> a[0] == a[0]
False
>>> a[1] == a[1]
True

The behaviour I was expecting does happen when one has a float array -- which is not my case, but I tried to test things:

>>> b = np.array([np.nan,1])
>>> b
array([ nan,   1.])
>>> np.equal(b,b)
array([False,  True], dtype=bool)

For the time being, my solution is:

>>> msk = np.array([ v==v for v in a ])
>>> msk
array([False,  True], dtype=bool)

Thanks.

[shoyer]

When I run your example with NumPy 1.11.1 I see:
FutureWarning: numpy equal will not check object identity in the future. The comparison did not return the same result as suggested by the identity (is)) and will change.

I'm not entirely sure when "the future" is, because I couldn't find the relevant issue on GitHub with a quick search. But we do definitely indeed want to fix this, because it's broken :).

[seberg]

Don't remember, I may have fixed this without any explicit issue open, might need to check the pull requests or just the code for when it was introduced.

[mattip]

The test np.equal(a,a) now returns array([False, True], dtype=bool), so I guess the future has arrived. Closing, please reopen if I am mistaken (not about the future, but about the issue at hand).