You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Mistral uses grouped-query attention with a configurable sliding window, RMSNorm, and a SiLU feed-forward network. Tests compare cached decoding against a full forward pass, including beyond the sliding window.
Gemma 2 uses offset RMSNorm, GELU, alternating local and global attention, and soft caps for both attention scores and output logits. Tests cover the attention window and check that cached decoding matches a full forward pass.
Gemma 3 text decoding reads nested text_config and explicit attention head dimensions. It applies query and key normalization, uses separate rotary position settings for local and global layers, and loads a separate output head when the checkpoint provides one.
Gemma 4 text decoding supports proportional rotary embeddings, shared key/value states, per-layer token inputs, and sparse mixture-of-experts routing. Tests cover the E2B configuration and mixed-bit MLX weights with packed 4-bit experts and an 8-bit router.
Checkpoint loading selects the decoder and tokenizer from the local Hugging Face configuration. The loader normalizes checkpoint tensor names across shards and can keep supported MLX weights packed during inference.
SentencePiece BPE and chat templates handle whitespace markers, UTF-8 byte fallback, and added tokens for supported Mistral and Gemma tokenizers. Gemma 2 and 3 fold the system message into the first user turn. Gemma 4 uses its own turn markers and follows the tokenizer's beginning-of-sequence rules to avoid inserting a duplicate token.
Verification scope covers text inference and architecture behavior with deterministic fixtures. Tests that compare against external Hugging Face checkpoints are optional.
Scientific Correctness & Data Integrity (SwiftStats, SwiftPreprocessing, SwiftDataFrame)
Corrected upper-tail Chi-square survival/CDF evaluation via regularized incomplete gamma precision fixes so chiSquareTest / goodness-of-fit $p$-values match SciPy (stat=156.274131, p=2.147307e-04, 65.67× faster at $N=100$ on clean idle hardware).
Implemented Royston AS R94 Shapiro-Wilk W and calibrated $p$-value transformation across $n \in [3, 5000]$ (verified exact W and $p$-value match at $n = 100, 1000, 5000$).
Calibrated Kolmogorov-Smirnov Lilliefors normal test (ksTest / normal parameters estimated from data) against empirical/regression-table Lilliefors critical-value curves (d=0.044160 at $N=200$, d=0.010317 at $N=2000$).
Replaced naive sum-of-squares computation in VarianceThreshold with numerically stable mean-shifted variance, avoiding catastrophic cancellation on high-offset features (1e12).
Added exact Bessel degrees-of-freedom handling and stable group variance accumulation in SelectKBest ANOVA $F$-test (f_classif).
Corrected RFE surviving feature index remapping back to original feature coordinates.
Enforced canonical non-decreasing multi-index ordering and bias-column semantics in PolynomialFeatures.
Enforced genuine strictly out-of-fold (OOF) K-fold cross-fitting inside TargetEncoder.fitTransform (DataFrame and numeric matrix paths) to eliminate target leakage, with smooth prior blending and minimum sample thresholds.
Updated DataFrame.unique to distinguish heterogeneous typed keys canonicalized across column storages instead of lossy string coercion.
Replaced naive TypedColumn.variance() / stdDev() paths with shifted/two-pass Welford formulation immune to $10^{12}$-offset cancellation.
Fixed CSV quoting and null-token round-trip contract across writer, eager parser, and streaming parser: unquoted empty/null tokens ("", NA, null, NaN, None) decode as nil, whereas quoted empty strings and quoted literal null tokens preserve their string values (wasQuoted propagated distinctly from escapedQuotesPresent; CSVWriter automatically quotes literals colliding with active null tokens).
Performance Optimizations (SwiftDataFrame)
Dedicated Compact Inner Hash Join (DataFrame+SIMDJoin.swift): Added a specialized .inner fast path without unnecessary BitSet / unmatched right-side tracking, promoted compact mixed-storage key handling, and consolidated output gathering:
medium-inner-join (100k rows): 11.155 ms → 2.185 ms (-80.4% vs pre-opt; -79.5% vs 3.12.010.635 ms).
mixed-medium-inner-join (100k rows): 13.856 ms → 2.209 ms (-84.1% vs pre-opt; -84.0% vs 3.12.013.808 ms).
Compact Mixed-Storage Gather (TypedColumn.swift): Routed gatheredScalar through gatheredNumeric where contiguous numeric storage applies, improving mixed-medium-filter from 0.429 ms to 0.298 ms (-30.5% vs pre-opt; -27.9% vs 3.12.0).
Fast Pre-Reserved CSV Serialization (CSVWriter.swift): Replaced per-cell boxing with direct ColumnAccessor extraction and pre-reserved single String output buffer (9.88 ms numeric = 4.94× faster than Pandas; 15.71 ms null tokens = 3.64×; 17.53 ms plain strings = 3.45× on clean idle runs; quoted null-token eager/streaming CSV reads 3.51 / 3.33 ms = 3.02–3.92× vs Pandas).
Verification, Conformance & Documentation
Audited standardized runs: certification (27/27 passed, max abs err 5.588e-09), dataframe-conformance (24/24 passed, max err 0.000e+00), scientific-workflow (4/4 passed, max err 1.421e-14), standard (14/14 passed), and migration (36/36 passed).
Restructured PERFORMANCE.md and ACCURACY.md into concise index landing pages backed by versioned archives (PERFORMANCE/{3.10.3..3.13.1}.md and ACCURACY/{3.10.3..3.13.1}.md), updated DocC articles, SwiftSciCLI (3.13.1), BenchmarkMain (v3.13.1), Engineering Handbook (70/70CookbookVerification recipes passed), 1,833 unit tests passing in Debug & Release (0 failures, 3 optional skipped), and 100.00% (2,551/2,551) heuristic public API documentation coverage.