Create ndarray (recarray?) subclass that can hold multiple Quantities of mixed units

[embray]

Over in this comment, @mhvk suggested that if we want an array-like container for Quantities of different units, if nothing else we could just use an ndarray of dtype=object to hold multiple Quantity objects. And in principle I'm fine with that as a solution.

However, I suggested in a followup that a subclass specially for this purpose might still be useful, if nothing else, for improved representation. For example, instead of an array of Quantities looking like:

[<Quantity 1.0 m>, <Quantity 2.0 km>, <Quantity 3.0 Jy>]

(which looks even worse when you go > 1 dimension), it might be nice to have something that reprs/prints like

[1.0 m, 2.0 m, 3.0 Jy]

etc. This would look especially nice if the repr is justified so that the columns line up :)

However, it turns out that a subclass for this purpose might be necessary even aside from aesthetic improvements, since it seems currently (at least with Numpy 1.9.1 which I've tested this on) it's not even possible to repr an array of Quantities:

In [23]: a = np.array([1 * u.km, 2 * u.J], dtype=object)

In [24]: print(a)
---------------------------------------------------------------------------
UnitsError                                Traceback (most recent call last)
<ipython-input-24-c5a4f3535135> in <module>()
----> 1 print(a)

C:\Python27\lib\site-packages\numpy\core\numeric.pyc in array_str(a, max_line_width, precision, suppress_small)
   1713
   1714     """
-> 1715     return array2string(a, max_line_width, precision, suppress_small, '
', "", str)
   1716
   1717 def set_string_function(f, repr=True):

C:\Python27\lib\site-packages\numpy\core\arrayprint.pyc in array2string(a, max_line_width, precision, suppress_small, separator, prefix, style, formatter)
    452     else:
    453         lst = _array2string(a, max_line_width, precision, suppress_small,
--> 454                             separator, prefix, formatter=formatter)
    455     return lst
    456

C:\Python27\lib\site-packages\numpy\core\arrayprint.pyc in _array2string(a, max_line_width, precision, suppress_small, separator, prefix, formatter)
    254
    255     formatdict = {'bool' : _boolFormatter,
--> 256                   'int' : IntegerFormat(data),
    257                   'float' : FloatFormat(data, precision, suppress_small),
    258                   'longfloat' : LongFloatFormat(precision),

C:\Python27\lib\site-packages\numpy\core\arrayprint.pyc in __init__(self, data)
    639     def __init__(self, data):
    640         try:
--> 641             max_str_len = max(len(str(maximum.reduce(data))),
    642                               len(str(minimum.reduce(data))))
    643             self.format = '%' + str(max_str_len) + 'd'

C:\MinGW\msys\1.0\home\embray\src\astropy\astropy\units\quantity.pyc in __array_prepare__(self, obj, context)
    293         # the unit the output from the ufunc will have.
    294         if function in UFUNC_HELPERS:
--> 295             converters, result_unit = UFUNC_HELPERS[function](function,
*units)
    296         else:
    297             raise TypeError("Unknown ufunc {0}.  Please raise issue on "

C:\MinGW\msys\1.0\home\embray\src\astropy\astropy\units\quantity_helper.pyc in helper_twoarg_comparison(f, unit1, unit2)
    305
    306 def helper_twoarg_comparison(f, unit1, unit2):
--> 307     converters, _ = get_converters_and_unit(f, unit1, unit2)
    308     return converters, None
    309

C:\MinGW\msys\1.0\home\embray\src\astropy\astropy\units\quantity_helper.pyc in get_converters_and_unit(f, *units)
    283                 "Can only apply '{0}' function to quantities "
    284                 "with compatible dimensions"
--> 285                 .format(f.__name__))
    286
    287         return converters, units[fixed]

UnitsError: Can only apply 'less' function to quantities with compatible dimensions

It seems that the Numpy array print code at some point tries ordering the values in the array, leading to this crash. This indicates maybe two things:

1. We could work around this specific error, I think, by making UnitsError a ValueError subclass.

2. Even if we fixed that specific issue it demonstrates that an array of Quantities can't be displays as well as one might like, and it might be worth fixing that.

A further question is why would we even need an array of Quantities of different units. I think if nothing else Quantity support in modeling may end up wanting something like this.

[mhvk]

Looking over old units issues, I saw this one. Am still somewhat intrigued, though as written I don't see how it could work, as all the mathematical operations don't make sense if different parts of a quantity have a different unit. It is similar to having an ndarray with mixed integer and float elements -- it just would not make much sense. Given that analogy, however, what might work is to have recarray subclass that has different units for its different columns (of course, this rapidly becomes a Table...). Indeed, if numpy proceeds with making it possible to define the unit on the dtype, then this would be nearly trivial to implement.

[mhvk]

(I removed "bug" since that aspect has been solved; it really is a "feature request" now)

[adrn]

One place where this concept would be useful is for making the Representation classes a bit nicer (they are a bit clunky right now IMO). If I recall, they currently store everything internally as Cartesian coordinates? But in principle, if I create a SphericalRepresentation it could avoid that conversion step unless necessary and store the position (length, angle, angle) as a single array (or just be an array/recarray subclass). Do you have any thoughts on this @mhvk or @eteq ?

[mhvk]

@adrn - the representation classes do store items as they are (i.e., SphericalRepresentation does store, radius, longitude, latitude). I do think there might be some benefits to using recarray instead, as it would remove some boiler-plate, but probably not a huge deal.

[adrn]

@mhvk oops, my mistake for mis-remembering...

[embray]

Well, don't use recarray specifically--that class is all but deprecated.

[mhvk]

I guess I meant an ndarray which uses a dtype with records. It might make sense to be able to associate different units with different records.

[adrn]

Another place where this might be useful is when storing phase-space information, e.g., position and velocity components, so this would be a useful feature!

[astrofrog]

Also for model parameters! (models have a way of accessing arrays of parameters but for models with units we just return a list for now)

[mhvk]

Just to note this is in the works, as it really helps deal with ERFA ufuncs that use/return structure dtype arguments (e.g., position and velocity).

[nstarman]

@mhvk, your PR closes this, right?

[mhvk]

Indeed. Now ensured this will be closed once #11775 is merged.