[C++] Cast between fixed size list type and variable size list

[asfimport]

Casting from fixed size list to variable size list could be possible, I think, but currently doesn't work:

>>> fixed_size = pa.array([[1, 2], [3, 4]], type=pa.list_(pa.int64(), 2))
>>> fixed_size.cast(pa.list_(pa.int64()))
...
ArrowNotImplementedError: Unsupported cast from fixed_size_list<item: int64>[2] to list using function cast_list

And in principle, a cast the other way around could also be possible if it is checked that each list has the correct length.

Reporter: Joris Van den Bossche / @jorisvandenbossche

PRs and other links:

• GitHub Pull Request #12517

_{Note: This issue was originally created as ARROW-15455. Please see the migration documentation for further details.}

[asfimport]

Eduardo Ponce / @edponce:
Also need to consider the scalar cases:

>>> fixed_size = pa.scalar([1, 2], type=pa.list_(pa.int64(), 2))
>>> fixed_size.cast(pa.list_(pa.int64()))
...
pyarrow.lib.ArrowNotImplementedError: casting scalars of type fixed_size_list<item: int64>[2] to type list<item: int64>

[asfimport]

Eduardo Ponce / @edponce:
Interesting to note that casting FixedSizeList to FixedSizeList works with Array but not with Scalar. The same behavior is observed for ListType.

# Works with Array
>>> fixed_size = pa.array([[1, 2]], type=pa.list_(pa.int64(), 2))
>>> fixed_size.cast(pa.list_(pa.int64(), 2))
<pyarrow.lib.FixedSizeListArray object at 0x1059e9f00>
[[1, 2]]

# Does not works with Scalar
>>> fixed_size = pa.scalar([1, 2], type=pa.list_(pa.int64(), 2))
>>> fixed_size.cast(pa.list_(pa.int64(), 2))
pyarrow.lib.ArrowNotImplementedError: casting scalars of type fixed_size_list<item: int64>[2] to type fixed_size_list<item: int64>[2]

[asfimport]

Jabari Booker / @JabariBooker:
It seem that you can't cast any lists with Scalar

 

#For Lists
>>> arrow_list = pa.scalar([1, 2])
>>> arrow_list.cast(pa.list_(pa.int64()))
pyarrow.lib.ArrowNotImplementedError: casting scalars of type list<item: int64> to type list<item: int64>

#For LargeLists
>>> arrow_list = pa.scalar([1, 2], pa.large_list(pa.int64()))
>>> arrow_list.cast(pa.large_list(pa.int64()))
pyarrow.lib.ArrowNotImplementedError: casting scalars of type large_list<item: int64> to type large_list<item: int64>

 

 

[asfimport]

Todd Farmer / @toddfarmer:
This issue was last updated over 90 days ago, which may be an indication it is no longer being actively worked. To better reflect the current state, the issue is being unassigned. Please feel free to re-take assignment of the issue if it is being actively worked, or if you plan to start that work soon.

[asfimport]

Jabari Booker / @JabariBooker:
Because the compute component of Arrow is not apart of core, we cannot call make calls to Cast() in Scalar::CastTo() . Therefore, we would need to reimplement casting to get Scalar::CastTo() work between list-type scalars. This is only impacts scalar.cast() in PyArrow. I was wondering if this effort seems at all worth it considering you simply use  pc.cast() anyways.

[asfimport]

David Li / @lidavidm:
It could be #ifdef'd. It would always work in Python since PyArrow requires ARROW_COMPUTE.

[asfimport]

Apache Arrow JIRA Bot:
This issue was last updated over 90 days ago, which may be an indication it is no longer being actively worked. To better reflect the current state, the issue is being unassigned per project policy. Please feel free to re-take assignment of the issue if it is being actively worked, or if you plan to start that work soon.