-
Notifications
You must be signed in to change notification settings - Fork 3.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Python][Doc] Clarify docstring of FixedSizeListArray.values that it ignores the offset #41672
Comments
Thanks for reporting this @pvardanis. This might be expected as |
@rok I'm using array_type = pa.list_(pa.list_(pa.int32(), list_size=2), list_size=2)
array = pa.array(
[[[1, 2], [3, 4]], [[5, 6], [7, 8]]],
type=array_type,
) |
@pvardanis I can confirm this is still behavior on main branch.
The reason this happens is that getting @jorisvandenbossche would you say this is approximately right? |
array.take([0]).values.values seems to be working, however I was hoping for a zero copy solution. array.slice(offset=0, length=1).values.values still returns the whole array. Also, I don't want to use |
@pvardanis i = 1
array.values.values.slice(offset=i * array.type.list_size, length=array.type.list_size)
<pyarrow.lib.Int32Array object at 0x71226477b2e0>
[
3,
4
] I think this should be zero-copy. |
@rok that works thanks! |
Great to hear! Closing the issue. |
@pvardanis watch out for the fact that And in |
We clarified the docstring of We should do the same for BTW, an important reason why the Of course, that reason is not important for |
I am going to reopen this issue and rephrase the title as a documentation issue to clarify the docstring. |
Technically the offsets exist the same way, but are implicit because they can be computed with Child arrays of nested layouts are required to keep the prefix-padding on the buffer when an upper level |
@jorisvandenbossche perhaps |
We clarified the docstring of
ListArray.values
a while ago to make it clear it ignores the offset: https://arrow.apache.org/docs/python/generated/pyarrow.ListArray.html#pyarrow.ListArray.valuesWe should do the same for
FixedSizeListArray.values
, as that right now is less explicit: https://arrow.apache.org/docs/python/generated/pyarrow.FixedSizeListArray.html#pyarrow.FixedSizeListArray.valuesOriginal report:
Describe the bug, including details regarding any error messages, version, and platform.
I have a
pa.FixedSizeListArray
that looks like this:I'm trying to flatten each row in the
pa.FixedSizeListArray
using.values
. Doingarray[0].values
, returns:but doing
array[0].values.values
returns the initial array flattened, not the one that I want:Weird thing is that
array[0].values.flatten()
returns what it's supposed to:However, I don't want to use
flatten()
since this discardsNull
values. Is this a bug or am I missing something?Component(s)
Python
The text was updated successfully, but these errors were encountered: