.real doesn't work on object arrays (Trac #1142)

[thouis]

Original ticket http://projects.scipy.org/numpy/ticket/1142 on 2009-06-19 by @inducer, assigned to unknown.

numpy could really do a better job here by propagating the .real to the elements of the object array:

Python 2.5.4 (r254:67916, Feb 18 2009, 03:00:47) 
[GCC 4.3.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as n
>>> n.__version__
'1.3.0'
>>> a = n.array(5+7j)
>>> b = n.array([a], dtype=object)
>>> b.real
array([(5+7j)], dtype=object)
>>> n.real(b)
array([(5+7j)], dtype=object)

---

Summary by @seberg:

Object arrays could call .real on all objects, but that would not return a view as typically promised by arr.real. It seems that moving forward here may require a new function that returns a copy?

[thouis]

Milestone changed to 1.4.0 by @cournape on 2009-11-25

[thouis]

Milestone changed to Unscheduled by @mwiebe on 2011-03-24

[charris]

Don't know if this makes sense or not. The real and imag methods return views, implementing them for object arrays would require generating new arrays. Common uses like assigning to the real part would also break. I'm inclined to close this.

[inducer]

Silently returning an unchanged array is, to my mind, entirely broken. Either throw an error or do something sensible.

[charris]

Good point. We should probably raise an error, probably for all non-numeric types.

[njsmith]

The current behaviour seems totally correct to me for numeric arrays -- in
particular that for non-complex arrays, .real returns 'self', and .imag
returns np.zeros_like(self) with the read-only flag set to True. However,
it is a bit weird that we extend this behaviour to non-numeric arrays --
e.g.

In [11]: a = np.array(["a", "b"])

In [12]: a.real
Out[12]:
array(['a', 'b'],
dtype='|S1')

In [13]: a.imag
Out[13]:
array(['', ''],
dtype='|S1')

Perhaps .real and .imag should only be defined if np.issubdtype(self.dtype,
np.number), and otherwise raise an error?

On Tue, Feb 18, 2014 at 2:43 PM, Andreas Klöckner
notifications@github.comwrote:

Silently returning an unchanged array is, to my mind, entirely broken.
Either throw an error or do something sensible.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/1740#issuecomment-35424726
.

Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

[njsmith]

Heh, you are more terse :-)

On Tue, Feb 18, 2014 at 2:52 PM, Charles Harris notifications@github.comwrote:

Good point. We should probably raise an error, probably for all
non-numeric types.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/1740#issuecomment-35425732
.

Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

[leewz]

I ran into this problem with sympy.mpmath.mpc. At least from my point of view as a user, it makes sense to ask for the real part, since mpc is a numeric type (from my perspective).

To implement them (inefficiently) for object arrays, can't a view can simply call the element's real and imag attributes each time?

a = n.array(5+7j)
b = n.array([a], dtype=object)
c = b.real
c[i] #equivalent to b[i].real

z = mpc(1+2j)
a = array([z])
r = a.real
r[0]      #calls `a[0].real`
r[0] = 1  #calls `a[0] = 1`
r[:] = 5  #iterates `a[i].real = 5`

(Related: Python objects in Numpy: compatibility issues with methods and functions)

[villirion]

In version 1.19.2 of numpy .real for objects still doesn't work and arrays in arrays are not interpreted but with two small functions we can easily solve the problem of .real for objects

import numpy as np
np.__version__

Out: '1.19.2'

a = np.array(5+7j)
b = np.array(6.2)
c = np.array("21")
d = np.array([a,b,a,c], dtype=object)
d = np.array([d,d], dtype=object)
d

Out: array([[array(5.+7.j), array(6.2), array(5.+7.j),
        array('21', dtype='<U2')],
       [array(5.+7.j), array(6.2), array(5.+7.j),
        array('21', dtype='<U2')]], dtype=object)

d.real

Out: array([[array(5.+7.j), array(6.2), array(5.+7.j),
        array('21', dtype='<U2')],
       [array(5.+7.j), array(6.2), array(5.+7.j),
        array('21', dtype='<U2')]], dtype=object)

d.imag

Out: array([[0, 0, 0, 0],
       [0, 0, 0, 0]], dtype=object)

def arrayObjToReal(arrayIn):
    if arrayIn.dtype=='O':      
        arrayWork = np.array([np.real(item) for item in arrayIn.reshape(-1) ],dtype=object)
        return arrayWork.reshape(arrayIn.shape)        
    else:
        return np.real(arrayIn) # or arrayIn.real

def arrayObjToImag(arrayIn):
    if arrayIn.dtype=='O':
        arrayWork = np.array([np.imag(item) for item in arrayIn.reshape(-1)],dtype=object)
        return arrayWork.reshape(arrayIn.shape) 
    else:
        return np.imag(arrayIn) # or arrayIn.imag

arrayObjToReal(d)

Out: array([[array(5.), array(6.2), array(5.), array('21', dtype='<U2')],
       [array(5.), array(6.2), array(5.), array('21', dtype='<U2')]],
      dtype=object)

arrayObjToImag(d)

Out: array([[array(7.), array(0.), array(7.), array('', dtype='<U2')],
       [array(7.), array(0.), array(7.), array('', dtype='<U2')]],
      dtype=object)

[WarrenWeckesser]

Another wrinkle with the array attributes .real and .imag is that they don't allow for custom data types that store complex numbers in a format other than (x + i*y). In the numtypes package, I created the data types polarcomplex64 and polarcomplex128 that store the complex numbers as (r, theta) (i.e. polar coordinates) instead of (x, y). The .real and .imag attributes of the scalars convert the internal representation to x or y on demand:

In [24]: import numpy as np

In [25]: from numtypes import polarcomplex128

In [26]: z = polarcomplex128(3+4j)

In [27]: z
Out[27]: polarcomplex128((5, 0.92729522))

The .real and .imag attributes are properties that compute the values on demand:

In [28]: z.real
Out[28]: 3.0000000000000004

In [29]: z.imag
Out[29]: 3.9999999999999996

The .real and .imag attributes of an array with this dtype do not do the right thing:

In [30]: a = np.array([z, -z, z + 1])

In [31]: a
Out[31]: 
array([polarcomplex128((5, 0.92729522)),
       polarcomplex128((-5, 0.92729522)),
       polarcomplex128((5.6568542, 0.78539816))], dtype=polarcomplex128)

In [32]: a.real  # incorrect
Out[32]: 
array([polarcomplex128((5, 0.92729522)),
       polarcomplex128((-5, 0.92729522)),
       polarcomplex128((5.6568542, 0.78539816))], dtype=polarcomplex128)

In [33]: a.imag  # incorrect
Out[33]: 
array([polarcomplex128((0, 0)), polarcomplex128((0, 0)),
       polarcomplex128((0, 0))], dtype=polarcomplex128)

[seberg]

Another wrinkle with the array attributes .real and .imag is that they don't allow for custom data types that store complex numbers in a format other than (x + i*y)

My current plan for solving this is the following:

1. Create a new custom slot (or two) on the DTypes to store an ArrayMethod that is called for real and complex (if not implemented an error is given).
2. Modify/fix the ArrayMethod.get_descriptors to include an npy_intp *view_offset parameter, that – if set – tells NumPy that the result is a view and what the offset for that view is. (Using an ArrayMethod is a bit overpowered here, but for symmetry is nice I think. Also at least .imag commonly does not return views.

Both changes are possible right now without any further modifications to the new DType system. Although, the second will create some code churn, but it should be straight forward.*

There could be a more generic approach (e.g. that would allow a Unit DType to add an arr.to_si()), but I think for the basic special methods/attributes using an explicit ArrayMethod on the DType is probably most straight forward.

---

As for .real and .imag, it would be OK to define this for objects by using the .imag and .real attributes of the underlying values I think. The main wrinkle is that for object DType we have no chance but to return a copy.

The way this is currently handled for .imag is that .imag returns a read-only array (since it is only 0s), we could extend that to .real as a generic behaviour (setting read-only) when a view is not possible. That at least pretends prevents the probably bigger issue of code doing arr.real += 1