np.average crashes for 1D decimal object array (now only without weights)

[NateyB]

I'm running macOS 10.12.3, Python 3.6.0, and NumPy 1.12.0.

When using the decimal package with NumPy, the arrays are of type 'object'. While most operations are performed flawlessly, calling np.average() with one of these arrays throws an error if weights are not provided (AttributeError: 'decimal.Decimal' object has no attribute 'dtype'), and a different error if weights are provided (AttributeError: 'bool' object has no attribute 'any'). Contrarily, calling np.mean() executes without error.

The source of the problem in both cases appears to be the assumption that the result of folding the input array results in a standard supported dtype, rather than being based on the operations available to the object type.

Therefore, I suspect that this issue will extend to any types which support the requisite numeric operations but are not natively supported by NumPy.

I resolved the weighted problem for my use case by changing the line if (scl == 0.0).any() (line 1138 in lib/function_base.py) to check if scl were an array first, and if not then removing the any() call, though I don't know if that solution is desirable or acceptable for the purposes of NumPy.

Sample input:

import numpy as np                                                              
import decimal as dc                                                            
                                                                                
values = np.array([dc.Decimal(x) for x in range(10)])                           
weights = np.array([dc.Decimal(x) for x in range(10)])                          
weights /= weights.sum()                                                        
                                                                                
print(np.mean(values))                                                          
print(np.average(values, weights=weights))                                      

Corresponding output:

4.5
Traceback (most recent call last):
  File "bugreport.py", line 9, in <module>
    print(np.average(values, weights=weights))
  File "/usr/local/lib/python3.6/site-packages/numpy/lib/function_base.py", line 1138, in average
    if (scl == 0.0).any():
AttributeError: 'bool' object has no attribute 'any'

[warut-vijit]

Yes, this seems like a bug and certainly doesn't reflected the intended behavior as described in the docs. Decimal arrays should be acceptable as arguments for this function.

[vbar]

Apparently the problem had regressed - I've just seen it with numpy 1.16.2 (Python 3.7 on Windows).

[eric-wieser]

The test is still there, and presumably passing in CI:

numpy/numpy/lib/tests/test_function_base.py

Lines 376 to 380 in dea1239

	def test_object_dtype(self):
	a = np.array([decimal.Decimal(x) for x in range(10)])
	w = np.array([decimal.Decimal(1) for _ in range(10)])
	w /= w.sum()
	assert_almost_equal(a.mean(0), average(a, weights=w))

Either your installation is a mixture of different versions, or you're seeing a different problem

[vbar]

I presume the test is for the case when weights are provided; my problem appears for average without weights:

Python 3.7.2 (tags/v3.7.2:9a3ffc0492, Dec 23 2018, 23:09:28) [MSC v.1916 64 bit
(AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> import decimal
>>> a = np.array([decimal.Decimal(x) for x in range(10)])
>>> np.average(a)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\XXX\AppData\Local\Programs\Python\Python37\lib\site-packages\numpy\lib\function_base.py", line 393, in average
    scl = avg.dtype.type(a.size/avg.size)
AttributeError: 'decimal.Decimal' object has no attribute 'dtype'
>>> np.__version__
'1.16.2'

A mixture of different versions is quite improbable - this is a new computer for a new project, where I've just installed Python and numpy...

[eric-wieser]

Turns out we didn't read the issue carefully enough:

throws an error if weights are not provided (AttributeError: 'decimal.Decimal' object has no attribute 'dtype'), and a different error if weights are provided (AttributeError: 'bool' object has no attribute 'any')

We only fixed one of these two problems.

Fixing this is another case where #13105 would help us...

[caioaamaral]

Any updates on this? I've found similar issue, but without the dc.Decimal(x)

x_model = np.average(x, weights=weights)  # where x and weights are a np.array of floats

Output error is (run at Python 2.7.12 and numpy 1:1.11.0-1ubuntu1):

 File "/usr/lib/python2.7/dist-packages/numpy/lib/function_base.py", line 946, in average
    if (scl == 0.0).any():
AttributeError: 'bool' object has no attribute 'any'

Upgrade numpy to 1.16 solves the issue for me.

[seberg]

The issue seems to be still relevant when weights are not given.