Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ARROW-15210: [Python] Pyarrow compute functions convert args with __arrow_array__. #12048

Closed
wants to merge 1 commit into from
Closed

ARROW-15210: [Python] Pyarrow compute functions convert args with __arrow_array__. #12048

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
3 changes: 2 additions & 1 deletion python/pyarrow/_compute.pyx
View file
Open in desktop
Expand Up @@ -416,7 +416,8 @@ cdef class MetaFunction(Function):

cdef _pack_compute_args(object values, vector[CDatum]* out):
for val in values:
if isinstance(val, (list, np.ndarray)):
if isinstance(val, (list, np.ndarray)) or \
hasattr(val, '__arrow_array__'):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we are adding this specific check, should we maybe also add a hasattr(val, "__array__") for also including "numpy-array-likes"?

(although that can maybe also decided in a separate JIRA)

It's just that the current set of types we try to convert to an array is somewhat arbitrary. List and np.ndarray are of course the logical things to support. But the current check would mean that, for example, a pd.Series or a generic sequence are not accepted (while asarray will accept those)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. I opened ARROW-15202 separately because pa.array also doesn't check for __array__. So not supporting __arrow_array__ here seemed like a bug to me, whereas supporting __array__ in general seemed more like a feature request.

val = lib.asarray(val)

if isinstance(val, Array):
Expand Down
9 changes: 9 additions & 0 deletions python/pyarrow/tests/test_compute.py
View file
Open in desktop
Expand Up @@ -293,6 +293,15 @@ def test_input_type_conversion():
"foo").to_pylist() == [True, False, None]


def test_input_array_conversion():
class convertible:
def __arrow_array__(self, type=None):
return pa.array(range(5), type)

assert pc.sum(np.arange(5)).as_py() == 10
assert pc.sum(convertible()).as_py() == 10


@pytest.mark.parametrize('arrow_type', numerical_arrow_types)
def test_sum_array(arrow_type):
arr = pa.array([1, 2, 3, 4], type=arrow_type)
Expand Down