Ability to choose data type for Categorical storage, U8, U16, instead of only U32 #16087
Labels
A-dtype-categorical
Area: categorical data type
enhancement
New feature or an improvement of an existing feature
Description
We can significantly decrease the memory usage of our dataset by using U8s instead of U32s for categorical encoding.
In a discussion on the Rust Polars Discord, it was mentioned that while the underlying Arrow dictionary type supports specifying the index type, although Polars currently has it hardcoded to U32.
The text was updated successfully, but these errors were encountered: