Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[SPARK-48019][SQL][FOLLOWUP] Use primitive arrays over object arrays …
…when nulls exist ### What changes were proposed in this pull request? This is a followup to #46254 . Instead of using object arrays when nulls are present, continue to use primitive arrays when appropriate. This PR sets the null bits appropriately for the primitive array copy. Primitive arrays are faster than object arrays and won't create unnecessary objects. ### Why are the changes needed? This will improve performance and memory usage, when nulls are present in the `ColumnarArray`. ### Does this PR introduce _any_ user-facing change? This is expected to be faster when copying `ColumnarArray`. ### How was this patch tested? Existing tests. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #46372 from gene-db/primitive-nulls. Authored-by: Gene Pang <gene.pang@databricks.com> Signed-off-by: Wenchen Fan <wenchen@databricks.com>
- Loading branch information