v0.2.0

v0.2.0

Sampling, streaming, and Qwen cache improvements since v0.1.0, with additional
ownership and profiling diagnostics. This remains a pre-1.0 source release;
model integrations are experimental, not a universal checkpoint runtime.

Added

• lm.SamplingOptions, lm.Sample, Model.GenerateWith, and
  Model.GenerateTokensWith: temperature, top-p filtering, and an MLX seed.
  Temperature zero uses the greedy path; top-p zero or one disables filtering.
  Options must be finite and in range. CLI flags are -temperature, -top-p,
  and -seed; an explicit -top-p requires positive temperature.
• lm.Stream and Model.Stream: caller-goroutine callbacks, partial results
  on callback errors, and incremental text decoding that withholds incomplete
  UTF-8 suffixes until completion or the final flush. Text output omits EOS;
  raw-token generation includes a matching stop token.
• Generation can return partial output with an error; always check the error
  before treating a result as complete. Prompt-mode cmd/generate writes text
  incrementally, followed by a timing line. Raw-token mode prints a final token
  list.
• Core CumsumAxis, Slice, and SliceUpdate with native/stub API parity.
• LiveArrays: process-wide owned, unclosed Go handle count, including
  idempotent/concurrent close and Qwen session cleanup checks.
• Synchronize: wait for submitted work on the selected default stream.
  The allocator regression test now synchronizes its measurement window;
  neither 8 MiB assertion was relaxed.

Improved

• Qwen preallocated KV buffers replace per-step whole-cache concatenation.
  The cache retains the base dtype and sessions evaluate buffers each step.
  Construct a cache with qwen2.NewKVCache(config, dtype); buffers grow lazily
  in 256-position blocks. Keep one cache per sequence and discard it after an
  error.
• Numeric readers use bulk copies after contiguous materialization; boolean
  readers retain element conversion. Returned slices remain Go-owned.
• Empty native output handles are marked closed and removed from live-handle
  accounting on operation failure.
• Homebrew cgo flags are centralized, with custom C/C++ prefix guidance.
• A top-level quickstart and separate training/verification guides make the
  existing supported workflows easier to find.

On the local Apple M3 Pro, the September 27 single-run GPU decode measurements
before/after the cache change were 100.5/101.5 tokens/s at a 48-token budget and
102.2/101.5 at 512 tokens, using the same prompt. These do not demonstrate a
meaningful speedup. They measure forward passes, not end-to-end throughput.

Installation

After the v0.2.0 tag is published:

brew install mlx-c
go get github.com/moncho/mlxgo@v0.2.0
go run -tags mlx ./your-app

Native support remains Apple Silicon macOS with cgo and the external MLX/mlx-c
libraries. The local validation baseline is Go 1.27.0, macOS 26.7, MLX 0.32.0,
and mlx-c 0.6.0_3. CI covers Go 1.26/1.27 on macOS 15, plus Linux stub builds.
See COMPATIBILITY.md for the support boundary and native dependency policy.

Validation

Local validation on 2026-10-04 passed:

• make release-check: stub/native tests and vet, the full native runtime
  suite, and the complete native race suite.
• All five core examples: smoke, linear, MLP, manual-gradient training, and
  autograd training. The real Qwen sampled CLI reproduced identical text in two
  fresh processes with temperature 0.8, top-p 0.9, and seed 42.
• Real Qwen golden verification: 32/32 reference tokens and maximum absolute
  prefill-logit error 0.03125, below the unchanged 0.25 limit.
• Real Qwen common-loader parity and temporary three-shard loading: 32/32
  reference tokens. The 200-token memory gate retained 44.2 -> 45.8 -> 46.0 MiB
  RSS, below the unchanged 64 MiB growth limit.
• DeepSeek's runtime suite with the existing local attention, compressed/shared
  attention, output/query projection, expert, and quantized-cache references
  enabled; the separate quantization suite also used its real activation oracle.

Real adapter-loader parity and saved training-checkpoint artifact comparison
were not enabled for this preparation. Reduced-fixture LoRA/training tests are
part of the runtime suite. Public CI does not download or run these local
real-weight fixtures. Existing numerical thresholds are unchanged.

Updating these files does not publish a tag or GitHub release. The exact
candidate must pass the complete CI matrix before tagging, then the tag
workflow reruns that matrix before publication.

Known Limits

• Sampling uses MLX's global RNG; interleaved random operations can change a
  seeded sequence. This is not a per-session random generator.
• Native work remains serialized on one OS thread. Stream callbacks run outside
  internal batches, but callers must not wrap them in worker code that blocks
  on another goroutine needing MLX. The default device is process-wide.
• LiveArrays is not a native-memory counter. Synchronize neither submits
  lazy graphs nor waits for unrelated streams; concurrent callers can still
  affect allocator measurements.
• Qwen support remains the documented standard BF16 architecture and q/v LoRA,
  not arbitrary models or QLoRA. DeepSeek remains experimental component and
  prepared-bundle support, not full released-checkpoint text inference.
• No DeepSeek module split, weight offloading, or quantized training is added.

The release contains Go source, not native libraries or model weights.
THIRD_PARTY_NOTICES.md records bundled upstream notices; model and native
dependency licenses apply separately.

v0.1.0

v0.1.0

First versioned release of mlxgo, Go bindings for MLX via mlx-c on Apple Silicon.
This is a pre-1.0 library release, not a stable universal model runtime.

Included

• CPU/Metal arrays, shape and math operations, autograd, optimizers, compilation,
  safetensors I/O, explicit ownership, dedicated-thread execution and batching.
• Qwen2.5-0.5B inference and q/v LoRA fine-tuning, including resumable training,
  adapter validation, a Go tokenizer and local single-file/sharded loading.
• Experimental DeepSeek components with independently checked real-weight
  samples, packed FP8/FP4 projections, quantized caches, opt-in sparse expert
  scheduling and prepared packed-bundle loading.
• Honest stub builds, native/stub API parity tests, CPU/GPU runtime regressions,
  race checks and runnable smoke examples.

Stabilization

• Fix Full(..., Float32) and other GPU-compatible fill dtypes by converting
  the scalar on CPU before broadcasting on the selected device. Integer fills
  do not lose precision through a float32 intermediate. Float64 output remains
  CPU-only. Tests cover dtypes, large integers, special floating values,
  scalar/empty shapes, errors, concurrent callers, compilation and autograd.
• Test Go 1.26 and 1.27 in CI, record native dependency versions, and require
  the complete CI matrix before publishing a GitHub release.
• Document the supported environment, pre-1.0 API policy and known limitations
  in COMPATIBILITY.md and the maintainer procedure in RELEASING.md.
• Include upstream license/provenance notices for the reduced Qwen tokenizer
  fixture alongside the existing DeepSeek reference notice.

Installation

brew install mlx-c
go get github.com/moncho/mlxgo@v0.1.0
go run -tags mlx ./your-app

Native execution requires an Apple Silicon Mac, cgo, and the Homebrew mlx-c
headers/libraries under /opt/homebrew. The local validation baseline is
Go 1.27.0, macOS 26.7, MLX 0.32.0 and mlx-c 0.6.0_3. CI also exercises macOS 15
and the Go 1.26/1.27 lines. Homebrew dependencies are external, not vendored.

Limits

• APIs are pre-1.0; higher-level model and quantization packages are experimental.
• This is not a loader for arbitrary model architectures or the full released
  DeepSeek-V4.1 checkpoint. DeepSeek samples do not produce pretrained text.
• Native calls share a worker thread. Callbacks and Batch bodies must not wait
  for another goroutine that needs MLX. Sparse scheduling performs host readback
  and is inference-only; dense execution remains the default.
• Packed weights do not imply faster CPU execution, quantized training,
  weight offloading or a guarantee that a checkpoint fits available memory.

The release distributes Go source, not native libraries, model weights or
platform binaries. Individual model licenses and native dependencies apply
separately from this repository's MIT license; see THIRD_PARTY_NOTICES.md.