Repository navigation
v0.14.4: CUDA gated-delta scan stability hardening
Fixes
- harden chunked
gated_delta_scanagainst suspected CUDA numerical instability by flooring multiplicative gate factors before log reconstruction, clamping recurrence exponents to a float32-safe range, and forcing float32 recurrence intermediates in the chunked recurrence - keep the Metal chunked path numerically aligned with existing FLA parity coverage while preserving the naive fallback via
scan_chunk_size: 0 - add a 50-step chunked gated-deltanet stability test to catch multi-step training failures that the one-shot parity tests miss
Verification
- Metal FLA parity remains unchanged in practice: single-layer
7.405877e-05, four-layer4.9114227e-05, max grad diff5.029142e-07 - Metal perf benchmark still shows a strong chunked win: naive
69,798 tok/s, chunked198,660 tok/s CGO_ENABLED=1 go test -tags mlx ./...passed on rerun/opt/homebrew/bin/golangci-lint runpassed
CUDA note
This patch addresses the highest-likelihood CUDA-specific numerical risk in the shipped chunked implementation, but CUDA hardware verification was not available locally. Please report any remaining issues at https://github.com/mrothroc/mixlab/issues.