[Python] Accept pd.NA as missing value in array constructor

[asfimport]

Currently we don't support using pandas.NA at all:

In [1]: import pyarrow as pa

In [2]: import pandas as pd

In [3]: pa.array([pd.NA, "A"])
---------------------------------------------------------------------------
ArrowInvalid                              Traceback (most recent call last)
<ipython-input-3-77e774fa8521> in <module>
----> 1 pa.array([pd.NA, "A"])

~/miniconda3/envs/fletcher/lib/python3.8/site-packages/pyarrow/array.pxi in pyarrow.lib.array()

~/miniconda3/envs/fletcher/lib/python3.8/site-packages/pyarrow/array.pxi in pyarrow.lib._sequence_to_array()

~/miniconda3/envs/fletcher/lib/python3.8/site-packages/pyarrow/error.pxi in pyarrow.lib.check_status()

ArrowInvalid: Could not convert <NA> with type NAType: did not recognize Python value type when inferring an Arrow data type

Reporter: Uwe Korn / @xhochy
Assignee: Wes McKinney / @wesm

Related issues:

• [C++/Python] Support necessary functionality to have an Arrow-string type in pandas (is depended upon by)

PRs and other links:

• GitHub Pull Request #7714

_{Note: This issue was originally created as ARROW-9407. Please see the migration documentation for further details.}

[asfimport]

Joris Van den Bossche / @jorisvandenbossche:
@wesm recently added support for pd.NA (ARROW-842, #7537), but apparently not yet for all cases.

First, from_pandas=True needs to be specified:

In [22]: pa.array(["A", pd.NA], from_pandas=True)                                                                                                                                                                  
Out[22]: 
<pyarrow.lib.StringArray object at 0x7ff2cf60fc48>
[
  "A",
  null
]

and thus doing like the above with a first typed element works. With pd.NA first it still doesn't work, so maybe that it is not yet correctly recognized as null where there is not yet an inferred type.

[asfimport]

Wes McKinney / @wesm:
Issue resolved by pull request 7714
#7714