-
Notifications
You must be signed in to change notification settings - Fork 674
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add GenericListViewArray
for ListView
& LargeListView
#5723
base: master
Are you sure you want to change the base?
Conversation
@tustvold If you have time, I would appreciate it if you could help me review this PR. Thanks! |
/// Creates a [`GenericListViewArray`] from an iterator of primitive values | ||
/// | ||
pub fn from_iter_primitive<T, P, I>(_iter: I) -> Self | ||
where | ||
T: ArrowPrimitiveType, | ||
P: IntoIterator<Item = Option<<T as ArrowPrimitiveType>::Native>>, | ||
I: IntoIterator<Item = Option<P>>, | ||
{ | ||
panic!("Not yet implemented, wait list view array builder to be implemented") | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Perhaps we could just not provide this for now
}; | ||
let iter = std::iter::repeat(size).take(value.len()); | ||
let mut offsets = Vec::with_capacity(iter.size_hint().0); | ||
offsets.push(OffsetSize::usize_as(0)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this correct?
.as_any() | ||
.downcast_ref::<Int32Array>() | ||
.unwrap() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
.as_any() | |
.downcast_ref::<Int32Array>() | |
.unwrap() | |
.as_primitive::<Int32Type>() |
And the same elsewhere
acc = acc.checked_add(size).expect("usize overflow"); | ||
offsets.push(OffsetSize::usize_as(acc)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
acc = acc.checked_add(size).expect("usize overflow"); | |
offsets.push(OffsetSize::usize_as(acc)); | |
offsets.push(OffsetSize::usize_as(acc)); | |
acc = acc.checked_add(size).expect("usize overflow"); |
.iter() | ||
.map(|x| x.map(|x| x.as_primitive::<Int32Type>().values().to_vec())) | ||
.collect(); | ||
assert_eq!(values, vec![Some(vec![1, 2, 3]), None, Some(vec![4, 5, 6])]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we add tests of the offsets and sizes buffers, I think they're currently not quite correct
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry for the delay, been swamped and then sick, but this is looking almost there
Thank you for taking the time to review my code despite being busy and unwell. |
Which issue does this PR close?
A part of #5501
Rationale for this change
What changes are included in this PR?
Add GenericListViewArray implement
Are there any user-facing changes?