Repository navigation
v0.1.1
v0.1.1: High-Performance Columnar Engine (Apache Arrow/Polars model)
Introduce a typed columnar storage architecture inspired by Apache Arrow and Polars to eliminate cell-level DataValue boxing, reduce dictionary allocations, and dramatically accelerate core DataFrame operations.
Highlights:
- Typed Columnar Storage: Added BitSet, AnyColumn protocol, and specialized columnar buffers (Float64Column, Int64Column, StringColumn, CategoricalColumn, BooleanColumn, DateColumn, AnyValueColumn) with zero-copy slicing.
- 64-bit Word Bitmap (BitSet): Zero-overhead null tracking and fast boolean mask evaluation with hardware popcount.
- High-Performance Joins: Replaced per-row dictionary construction with integer index gathering and slice(atOptional:) columnar slicing (~3x faster left joins: 19.59ms -> 6.95ms on 40,000 rows).
- Vectorized GroupBy: Single-pass dense Int32 group ID assignment and scatter aggregation directly into contiguous accumulation buffers (43% faster: 12.01ms -> 6.79ms).
- Direct Primitive Sorting: Fast paths for Double, categorical code ranks, timestamps, and strings directly sorting raw index buffers.
- Streamlined CSV Parser: File size-based capacity pre-allocation, quote-free line splitting, UTF-8 BOM auto-detection, and zero-allocation Substring-based numeric parsing (130ms -> 84.39ms).
- 100% Backward Compatibility: All 63 existing unit tests pass with zero compiler warnings."