I think you could avoid a lot of this copy / paste by using a StringViewBuilder with the append_block and append_view_unchecked methods
https://docs.rs/arrow/latest/arrow/array/type.StringViewBuilder.html#method.append_block
That being said, I do think it could be slightly slower than this implementation because it would have to re-check the length
It almost seems like what we want is some sort of API on StringViewArray itself, similar to https://docs.rs/arrow/latest/arrow/array/struct.PrimitiveArray.html#method.unary
So this code could be written something like
let new_array = orig_array.map_values(convert)
That would also let us do potentially crazy things like reuse the buffer allocations and modify the values in place if they weren't shared 🤔
If that makes sense to you I can file a ticket in arrow-rs perhaps.
Originally posted by @alamb in #21980 (comment)
I think you could avoid a lot of this copy / paste by using a
StringViewBuilderwith theappend_blockandappend_view_uncheckedmethodshttps://docs.rs/arrow/latest/arrow/array/type.StringViewBuilder.html#method.append_block
That being said, I do think it could be slightly slower than this implementation because it would have to re-check the length
It almost seems like what we want is some sort of API on
StringViewArrayitself, similar to https://docs.rs/arrow/latest/arrow/array/struct.PrimitiveArray.html#method.unarySo this code could be written something like
That would also let us do potentially crazy things like reuse the buffer allocations and modify the values in place if they weren't shared 🤔
If that makes sense to you I can file a ticket in arrow-rs perhaps.
Originally posted by @alamb in #21980 (comment)