[Python] KeyError when using compute function month_interval_between()

[vcfgv]

Describe the bug, including details regarding any error messages, version, and platform.

Code is shown as below. I have no idea of the cause of this error message.

import pyarrow as pa
import pyarrow.compute as pc

array_0 = pc.strptime(
    pa.array(["2021-01-01 00:00:00", "2022-01-01 00:00:00"]),
    format="%Y-%m-%d %H:%M:%S",
    unit="ns"
)

array_1 = pc.strptime(
    pa.array(["2021-01-01 00:00:00", "2022-01-01 00:00:00"]),
    format="%Y-%m-%d %H:%M:%S",
    unit="ns"
)

pc.month_interval_between(array_0, array_1)

Error Message

KeyError Traceback (most recent call last)
/tmp/ipykernel_314034/605650526.py in
11 )
12
---> 13 pc.month_interval_between(array_0, array_1)

~/miniconda3/lib/python3.7/site-packages/pyarrow/compute.py in wrapper(memory_pool, *args)
240 if args and isinstance(args[0], Expression):
241 return Expression._call(func_name, list(args))
--> 242 return func.call(args, None, memory_pool)
243 else:
244 def wrapper(*args, memory_pool=None, options=None, **kwargs):

~/miniconda3/lib/python3.7/site-packages/pyarrow/_compute.pyx in pyarrow._compute.Function.call()

~/miniconda3/lib/python3.7/site-packages/pyarrow/array.pxi in pyarrow.lib.wrap_datum()

~/miniconda3/lib/python3.7/site-packages/pyarrow/public-api.pxi in pyarrow.lib.pyarrow_wrap_array()

~/miniconda3/lib/python3.7/site-packages/pyarrow/array.pxi in pyarrow.lib.get_array_class_from_type()

KeyError: 21

Version
python: 3.7.9
pyarrow: 12.0.0

Additional Information

• When using pyarrow 6.0.1, the return value of pc.month_interval_between(array_0, array_1) is a ChunkedArray but calling combine_chunks() will also get a KeyError(21)

chunked_arr = pc.month_interval_between(array_0, array_1)
chunked_arr.combine_chunks()

• Changing datetime string does not seem helpful.
• Changing month_interval_between() to month_day_nano_interval_between() will not get the aforementioned KeyError.

Component(s)

Python

[jorisvandenbossche]

@vcfgv thanks for opening the issue! You are running into a limitation of the pyarrow python bindings (although the error message is not very clear about what is going on ..).
The "month_interval_between" kernel returns an array with the interval type with the "year_month" unit. And currently, the python bindings only support the "month_day_nano" unit.

If you use the "month_day_nano_interval_between" kernel instead, you do get a result, using your example data:

>>> pc.month_day_nano_interval_between(array_0, array_1)
<pyarrow.lib.MonthDayNanoIntervalArray object at 0x7f41b7c9b5e0>
[
  0M0d0ns,
  0M0d0ns
]

[jorisvandenbossche]

In practice, we can consider this as a duplicate (or consequence) of #29828

[vcfgv]

Thank you for your informative response. @jorisvandenbossche