[Python] Empty numpy arrays with shape cannot be deserialized

[asfimport]

In PyArrow 0.16.0, when we try to deserialize a serialized empty Numpy Array with shape, for instance, np.array([[], []]), an ArrowInvalid is raised.

Code reproducing this error:

import numpy as np
import pyarrow
arr = np.array([[], []])
pyarrow.deserialize(pyarrow.serialize(arr).to_buffer())  # this line cannot work

and the error stack is

Traceback (most recent call last):
  File "/Users/wenjun/miniconda3/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3326, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-4-0ace9226dd72>", line 1, in <module>
    pyarrow.deserialize(pyarrow.serialize(arr).to_buffer())
  File "pyarrow/serialization.pxi", line 476, in pyarrow.lib.deserialize
  File "pyarrow/serialization.pxi", line 438, in pyarrow.lib.deserialize_from
  File "pyarrow/serialization.pxi", line 414, in pyarrow.lib.read_serialized
  File "pyarrow/error.pxi", line 84, in pyarrow.lib.check_status
pyarrow.lib.ArrowInvalid: strides must not involve buffer over run

The same code works in PyArrow 0.15.x

Reporter: Wenjun Si / @wjsi
Assignee: Wenjun Si / @wjsi

Related issues:

• [Python] Error serializing empty pandas DataFrame with pyarrow (relates to)

PRs and other links:

• GitHub Pull Request #6624

_{Note: This issue was originally created as ARROW-8122. Please see the migration documentation for further details.}

[asfimport]

Joris Van den Bossche / @jorisvandenbossche:
Thanks for the report and PR!

Related issue (reported as serialization of empty dataframe): ARROW-7996

[asfimport]

Wes McKinney / @wesm:
Issue resolved by pull request 6624
#6624