Repository navigation
mlx-reason-ct 0.4.0
Fixes
Process memory no longer balloons during one-shot reports. generate_report, run and the CLI now grow the KV cache in preallocated 256-token chunks instead of reallocating it for every token at the ~14k-token CT context. Peak MLX memory for a float32 chest report is about 13.7 GB.
New
--precision bfloat16_fast is the fastest profile. It uses BF16 operands for the CT and prompt and FP32 for generated tokens, the LM head, norms, attention and caches. Its text can differ from float32 where the two most likely tokens nearly tie.
Faster
float32 reports take about 0.6x the time of 0.3.6, with bit-identical output (chest report, M5 Max). Decoder weights are held as their exact BF16 source values, and a matching kernel halves the bytes read per generated token.