ufuncs on object arrays should call __array_ufunc__ if it exists on the wrapped objects

[shoyer]

Reproducing code example:

Here's my dummy class that implement __array_ufunc__ for ufuncs and arithmetic:

import numpy as np

class Wrapper(np.lib.mixins.NDArrayOperatorsMixin):
  def __array_ufunc__(self, ufunc, method, *inputs, **kwargs):
    return 'yes'

It works fine on its own, and scalar arithmetic works fine when wrapped in a dtype=object numpy.array. But calling ufuncs on the object array raises a strange TypeError:

print(Wrapper() + 1)  # 'yes'
print(np.sqrt(Wrapper()))  # 'yes'
print(np.array(Wrapper()) + 1)  # 'yes'
print(np.sqrt(np.array(Wrapper())))  # errors

Error message:

AttributeError                            Traceback (most recent call last)
AttributeError: 'Wrapper' object has no attribute 'sqrt'

The above exception was the direct cause of the following exception:

TypeError                                 Traceback (most recent call last)
<ipython-input-59-ace91090b167> in <module>()
      8 print(np.sqrt(Wrapper()))  # 'yes'
      9 print(np.array(Wrapper()) + 1)  # 'yes'
---> 10 print(np.sqrt(np.array(Wrapper())))  # errors

TypeError: loop of ufunc does not support argument 0 of type Wrapper which has no callable sqrt method

I'm not sure what exactly is going on, but it seems that we don't call actual ufuncs on the elements of dtype=object arrays, and instead only look for methods.

Numpy/Python version information:

1.17.5 3.6.9 (default, Nov 7 2019, 10:44:02)
[GCC 8.3.0]

[seberg]

Note that np.add(np.array(Wrapper()), 1) works. What is going on is simple, the addition example calls: element + 1 for every single element. And the element is an array that defines __add__ correctly. However, each element (your Wrapped() class) does not define .sqrt(). So the UFunc call ends up trying element.sqrt() which fails.

Now, you could add a "fallback" or initial path, that tests whether the "element" is an array like and thus might now know how to apply the UFunc on it....

But: I am not sure we should... Note that you get the same result for NumPy arrays (with the exception that array.sqrt() happens to be defined, so you have to use a different method). Also looking up __array_ufunc__ on every single element might be slow? (OTOH, you could cache it I suppose). My gut feeling is that the current behaviour is correct, although a bit confusing. Do you have a specific use-case?

[shoyer]

OK, probably the simplest reproduction is:

>>> np.sin(np.array([1, 2], dtype=object))
TypeError: loop of ufunc does not support argument 0 of type int which has no callable sin method

Of course I have a use-case for this, but it's pretty non-canonica). I'm not entirely sure it's worth the trouble :)

[seberg]

There is another problem I ran into while trying isposinf is currently not defined for "object". So although your Wrapped() object could handle it fine, NumPy will think there is

Well, it was possibly a knee jerk reaction, but I might need a bit more convincing :). Although, I admit that an object exporting __array_ufunc__ (lets include arrays in this) obviously thinks it knows how to apply a ufunc. So aside from the fact that our machinery is a bit ill equipped for it, there is at least no consistency issue...

[larsbuntemeyer]

I have an issue that might be similar. I was trying to get cftime objects to accept np.timedelta64 objects to use for arithmetics (it is implemented for the underlying datetime.timedelta item anyway). However, i can't get it to run with arrays. I think i can boile it down to a simliar issue, so assume dummy being a calendar class that knows how to handle np.timedelta64:

import numpy as np

class dummy(np.lib.mixins.NDArrayOperatorsMixin):
  def __array_ufunc__(self, ufunc, method, *inputs, **kwargs):
    return 'GO!'

print(dummy() + 1) # GO!
print(dummy() + np.float64(1)) # GO!
print(dummy() + np.timedelta64(1, 'D')) # GO!
print(np.add(dummy(), np.timedelta64(1, 'D'))) # GO!
print(np.array(dummy()) + np.float64(1)) # GO!
print(np.array(dummy()) + np.timedelta64(1, 'D')) # error

I am puzzled since when it works with np.add it also should work on object arrays? Why does it work with np.float64 but not np.timedelta64?

[seberg]

The one is also because the object loop exists but isn't picked:

UFuncTypeError: ufunc 'add' cannot use operands with types dtype('O') and dtype('<m8')

That is a small bug in the legacy promotion step (object should win out against datetimes).