ArrowMetal 0.1.0

Everything below is in 0.1.0, the first public release.

Core

• Metal shared-memory Arrow buffers (page aligned, pooled) and primitive/boolean arrays.
• Kernels: sum/min/max/mean, compare, add/sub/mul/div (vectorised, defined integer division by zero),
  filter (one command buffer, GPU scan) and fused filter(where:), take, cast, slice, boolean and/or/not/count/any/all.
• Float64 compare/min/max/filter/take/slice on the GPU via order-preserving bit patterns; NaN semantics.
• GroupBy over dense integer keys: count, sum, mean, min, max (privatised and device-atomic paths), plus a
  sort-based segmented path with no atomics — sumDouble/meanDouble/sumFloatAsDouble/meanFloat and
  min64/max64 — which covers the Float64 and 64-bit min/max cases 32-bit atomics cannot express.
• The key-value-to-dense-id mapping reads an integer key column as the type it is rather than widening it
  to int64 first, and its three dispatches share one command buffer; hash_mean reuses the counts the
  accumulation already produced and divides on the GPU, and hash_sum returns the accumulator's own
  buffer with a GPU-built validity bitmap instead of a host loop. Same ids, same answers to the bit;
  sum by int32 key at 50M rows and a thousand groups went 8.75 -> 4.81 ms and mean 9.78 -> 4.80 in
  the A/B run recorded in docs/TO_IMPROVE.md; the published matrix reads 4.89 and 4.91 ms for those cells.
• MetalRecordBatch with filter/take/slice/selecting; struct (+s) C Data import/export; ArrowArrayStream import.
• Batched execution (MetalContext.batch { }) and its non-blocking form: batchAsync (Swift async
  and completion-handler), with MetalArray.sumAsync/meanAsync for scalars, so the calling thread is
  free while the GPU works.
• Arrow C Data Interface and C Device Data Interface (ARROW_DEVICE_METAL) import and export.

Strings and sorting

• MetalStringArray (utf8): byte/char length, equals/starts_with/ends_with/contains, MurmurHash3, GPU filter/take,
  GPU dictionary encoding (hash, argsort, byte-comparing boundaries, rank scan and gather; collisions detected
  and re-hashed, with the host path as the final fallback); import/export through the C Data Interface
  (large_utf8 narrowed on import).
• GPU LSD radix sort: argsort, sorted, MetalRecordBatch.sorted(by:); stable, nulls last by default
  (null_placement puts them at either end), IEEE total order.
  The nulls are partitioned out before the sort rather than lifted out of the permutation afterwards, and
  sorted() rebuilds the values from the sort's own keys instead of gathering them through it.
• topK for any k: a GPU radix select (digit histogram over the order-preserving key, one compaction pass
  keeping only the rows that can still win, a refinement round, then a bitonic or radix ordering) reads the
  column twice whatever k is, and matches the full sort index for index. The per-threadgroup selection
  (threshold plus a bitonic compaction in threadgroup memory) still serves small inputs at k <= 1024, and
  the sort covers k near n and the case where fewer than k rows are non-null.
• kthElement(k, largest:): the exact k-th smallest or largest value, by the same selection with the winners
  counted but never written. quantile and approximateMedian run on it instead of a full sort.

Temporal, binary and dictionary types

• MetalTemporalArray: date32/date64, time32/time64, timestamp (unit + optional timezone) and duration,
  forwarding compare/filter/take/slice/min/max/sort/argsort to the int32 or int64 array underneath.
• GPU calendar fields in UTC (year, month, day, dayOfWeek, hour, minute, second), plus
  toDate32() and castUnit(to:); civil-from-days arithmetic, correct for negative epochs.
• binary and large_binary share the utf8 layout (MetalStringArray.isBinary, exported as "z").
• Dictionary-encoded arrays import as int32 codes plus a value array; selection runs on the codes,
  decode() materialises with take, and export writes schema.dictionary / array.dictionary.
• C ABI: am_temporal_extract, am_temporal_cast_unit, am_dictionary_decode; Python year() ...
  second(), cast_unit(), decode() and temporal/binary/dictionary types on MetalArray.type.

Execution model

• MetalContext.batch { }: one command buffer per chain; kernels read lengths from device buffers so pending
  filter results chain without a CPU sync; pool parks buffers while a batch is open.
• Software IEEE-754 Float64 on the GPU: add/sub/mul/div are correctly rounded and bit-exact
  against Swift's Double (DoubleMathTests); sum uses the same adder but reassociates over
  threadgroups, so it is checked to a tolerance rather than bit for bit.

Numerics

• Checked arithmetic: add_checked … power_checked, negate/abs/sqrt/ln/log10/log2/log1p,
  the checked shifts, logb_checked, and checked cumulative_sum/cumulative_prod/pairwise_diff.
  Each is the unchecked kernel plus a read-only check pass in the same command buffer, so the values are
  bit-identical and the cost is one GPU round trip; a failure names the Arrow message and the first row.
• Trigonometry: the twelve trigonometric and hyperbolic functions, their seven _checked twins and
  atan2 — Metal's library functions for float32 (hyperbolics rewritten from well-conditioned
  identities), a software binary64 implementation for float64. Measured within 5 ulp of the host libm
  over 1,000,003 random arguments per function (float64 worst 5, tan; float32 worst 4 — TrigTests).
  Above |x| ≈ 5e13 the argument reduction degrades in step with the argument's own ulp (7 ulp measured
  at 5e13) and |x| ≥ 2^62 returns NaN — the one documented difference from libm.
• The remaining element-wise math: expm1, log1p, logb, hypot, round_to_multiple, round_binary
  and round with all ten Arrow round modes and any ndigits.
• Float64 sqrt/exp/ln/log2/log10/power (and their _checked twins) in real software
  binary64, not a float evaluation widened back: sqrt is correctly rounded — bit-identical to
  Foundation over 10^6 random bit patterns plus the extremes — and the rest measure 1 ulp over 10^6
  inputs each across the whole domain, against a 2-ulp bound the tests assert.
  power on a float64 column is new; it used to raise. The trade is throughput, stated in
  docs/BENCHMARKS.md: a binary64 logarithm costs about 25x what the seven-digit one did.
• Software binary64 arithmetic made faster without losing a bit: clz normalisation instead of shift
  loops, four 32x32 partial products instead of an emulated 64x64, and a Newton reciprocal with an exact
  128-bit remainder correction instead of a 57-step restoring division. Float64 add and multiply
  now run at about 390 GB/s at 50M rows, the fastest these single-pass float64 rows reach, and divide within 15% of it
  (331 GB/s) — docs/DESIGN.md.
• Float classification (is_nan, is_finite, is_inf) as raw bit-pattern tests, and the boolean
  operators the bitmap family lacked: xor, and_not, and_not_kleene.
• Statistical aggregates skew, kurtosis and tdigest (GPU sort, host centroid merge), with grouped
  forms of all three.

Grouped aggregation and windows

• GroupByKeys: group-by over arbitrary key columns — sparse or negative integers, floats (with
  -0.0 == 0.0 and one NaN group), booleans, temporal values, utf8, binary, dictionary and decimal, and
  several columns folded pairwise into an injective composite. A range path (mark, scan, rank) for narrow
  integer-like columns and the dictionary-encode sort path for everything else.
• All 24 Arrow hash_* names against those ids: the fused hash_min_max, hash_count_all,
  hash_first/hash_last/hash_first_last, hash_one, hash_list, hash_distinct,
  hash_approximate_median and hash_quantile (exact, from a segmented sort), hash_product,
  hash_variance/hash_stddev, hash_skew/hash_kurtosis, hash_tdigest and hash_pivot_wider.
• hash_count_distinct is a GPU hash set over the (key, value) pair — one insert pass over the rows,
  then a histogram over the group ids of the occupied slots — instead of a dictionary encoding, a packed
  int64 column and a unique() over it. 7.1 ms at 10M rows and 41.8 ms at 50M for a thousand groups,
  and 7.3 ms at 10M for a hundred thousand — 60.7 ms at 50M rows and ten million groups is the worst
  case (docs/TO_IMPROVE.md).
• Several integer key columns whose ranges multiply out to at most 2^24 (and at most the row count) are
  packed into one key in a single pass instead of folded pairwise through a range encoding each. The
  dense ids are the fold's own, value for value, so the group order is unchanged.
• Window and ordering functions: rank, dense_rank, row_number, rank_quantile, rank_normal,
  winsorize, shift, rolling sum/mean/min/max, cumulative_prod/cumulative_mean,
  pairwise_diff, multi-column lexsort_indices, partition_nth_indices, inverse_permutation,
  scatter, unique, value_counts, count_all, true_unless_null, first_last and random
  (Philox4x32-10, seed-only stream).

Strings and Unicode

• The utf8_is_* predicate family plus ascii_is_printable, ascii_is_title and string_is_ascii. Each
  answers every row on the GPU and re-decides on the host only the rows carrying a byte >= 0x80, so an
  all-ASCII column never leaves the device.
• Case and layout transforms: ascii_title, utf8_capitalize, utf8_title, utf8_swapcase,
  utf8_center, utf8_zero_fill, utf8_replace_slice, binary_replace_slice, the utf8_trim* family
  against the full Unicode whitespace class, and utf8_normalize (NFC/NFKC/NFD/NFKD, host-side).
• extract_regex_span, binary_join over list<utf8>, and string/binary value sets for is_in and
  index_in through the GPU string hash table.

Temporal, timezones and the rest of the type matrix

• week with all its WeekOptions, ...