Skip to content

ArrowMetal 0.6.1

Latest

Choose a tag to compare

@singhpratech singhpratech released this 08 Nov 13:40
· 4 commits to main since this release

Everything below is new in 0.6.1. On M1-generation GPUs, 0.6.0's sorts of 64-bit keys and its privatised
group-bys failed to create their pipelines; the fix is the first outside contribution, by d-polikhranidi
(#1, #2). The Polars engine runs on polars 2.1.0, and the polars extra installs polars>=1.44,<2.2.

  • Sorts and group-bys compile on M1 GPUs. The GPU compiler on an M1 Max (macOS 27.2) aborts on a ulong joined
    with shifts from two threadgroup atomic_uint loads, which radix_histogram, gb_accumulate_priv and am_gb_priv
    did, so every sort of 64-bit keys (Int64, Double, temporal, utf8, binary) and those group-bys failed to create their
    pipelines there. The three join with as_type<ulong>(uint2(lo, hi)) instead (#1, d-polikhranidi's #2), and so does
    gs_b_priv, whose four such loads passed on that machine but had the same shape. GitHub's macos-15 runner
    refused the same construct: its conformance subset now builds every kernel its 2,000 sampled cases ask for
    (1,868 pass, 81 documented, 0 pipeline failures on 2026-11-08; 496 pass and 1,491 pipeline skips on 2026-11-06).
  • Polars 2.1.0. MetalEngine runs on polars 2.1.0 (IR (16, 0)); TESTED_POLARS adds 2.1.0 and the
    polars extra installs polars>=1.44,<2.2. Polars 2.1 shows engines a join holding a predicate over
    both sides (join(...).filter(pl.col("x") < pl.col("y")) becomes one inner join of kind
    ("Inner", "fused_predicate")): the engine runs it as the join, then the filter; other join kinds
    with a predicate stay with Polars (on 2.0.0 Polars kept such a join from engines). With the 2.0.0
    crossover table, the default on 2.1.0 takes 52 of 107 benchmarked cases at 50,000,000 rows, each
    1.31x to 10.31x faster than the faster Polars 2.1.0 engine, none behind
    (Benchmarks/results/polars_engine_bench_2026-11-06-polars21.csv). The engine conformance grid,
    run on 2.0.0 and 2.1.0 for the first time, records the streaming engine's element-wise division of
    frames under 100 rows as a documented divergence (36 cases); 0 unclassified on 1.44.1, 2.0.0 and
    2.1.0 (docs/FINDINGS.md, Round 24).