Open
Description
In #473 we added basic functionality for adaptive array builder but currently when upgrading the dictionary, we copy all the values.
Ideally we'd just be able to get access to the underlying key builder of the dictionary builder, finish/cast it to the new index type, and then create a new dictionary builder with the same values/internal state + the new key builder. As far as I know there's not currently a way to do this in arrow-rs.
TODO
- confirm with arrow-rs community if this is possible
- open any necessary issues in arrow-rs for this capability
- implement changes in arrow-rs
- use optimized implementation of dictionary builder conversion in dictionary upgrade.
Metadata
Metadata
Assignees
Type
Projects
Status
No status