Skip to content

v0.1.0 — Experimental BF16 inference milestone

Pre-release
Pre-release

Choose a tag to compare

View all tags
@popbones popbones released this 27 Oct 05:28
v0.1.0
0e38b4c

First experimental public milestone for running the official LiveMem-RL BF16 checkpoint on Apple Silicon with MLX and a custom Metal GDN2 kernel.

Includes Hugging Face revision-pinned loading, local/offline checkpoints, a Python API and CLI, explicit per-session memory state, bounded attention KV, and experimental quantization tooling. The official weights are downloaded separately from chen-l/LiveMem-RL; no weights are attached or bundled.

Validation

  • 58 source tests and 58 installed-wheel tests passed for the release build.
  • The inference modules match the development wheel used for the full-checkpoint smoke tests; only the package version changed.
  • Official BF16 and development Q8/Q4 checkpoints completed 12 within-precision comparisons against the existing MLX runtime. Generated token IDs, retained KV positions, and saved prefill logits matched exactly.
  • Distribution privacy/secret-pattern scans and license packaging checks passed.

See checkpoint validation for settings, provenance and limits.

Install

On an Apple Silicon Mac with Python 3.11 or later:

python -m pip install "git+https://github.com/popbones/livemem-mlx.git@v0.1.0"
livemem-mlx generate --model chen-l/LiveMem-RL --prompt "Hello" --max-tokens 64

Alternatively, install the attached wheel. SHA-256 checksums for the wheel and source archive are attached. This package is not published to PyPI.

Scope and limitations

This is an experimental BF16 inference milestone. Quantized weights remain development artifacts: none are published, attached, or approved for release. The included conversion tooling is experimental.

Full-model CUDA/vLLM parity, broad memory quality, long-running stability, and quantized quality relative to BF16 are not established. Session state is explicit and is not automatically shared across requests. See the repository's runtime, evidence and licensing-audit documents.

Package terms are Apache-2.0 AND MIT, with upstream notices included. This is an independent implementation, not an official LiveMem release or endorsement.

Assets 5
Loading