sys1rust 0.1.0

sys1rust runs Laya's System 1 decision models on an Apple silicon Mac's GPU. It is written in Rust on MLX, with custom Metal kernels and no Python at run time.

Send text and questions to its /v1/systemone API to get choices, scores and yes probabilities. The server accepts the same requests as laya serve.

Install

Give a coding agent this prompt:

Set up sys1rust on my Mac from https://github.com/krishhgg/sys1rust

Or install from a native Apple silicon terminal:

curl -fsSL https://raw.githubusercontent.com/krishhgg/sys1rust/main/install.sh | sh
~/.local/bin/sys1rust serve

The first start downloads the default typed-decisions model, 846 MB. The README has an example request and the setup, use and development instructions for coding agents.

Bundles

• sys1rust-0.1.0-macos14-arm64.tar.gz supports macOS 14.0 through 26.1.
• sys1rust-0.1.0-macos26-arm64.tar.gz supports macOS 26.2 and later.
• SHA256SUMS contains the hashes of both bundles. The installer checks the download against this file and selects the bundle for your macOS version.

Both bundles include MLX 0.32.2 and their license notices. Installation needs no Homebrew, sudo, Python or API token. Use --service to run at login and restart after a crash. Updates retain the previous version, and uninstall keeps the downloaded models.

The included models are typed-decisions, multilingual and english. The server binds to 127.0.0.1 by default.

Performance

On a base M5 with macOS 26.2 and the macOS 26 bundle, the typed-decisions model takes a median 15.5 ms for a request with one question and a 128-token state. The measured workloads are up to 4.1x faster than stock laya serve. Performance on other Mac chips has not been benchmarked.

See the README for setup and the results for the measurements and their limits.

sys1rust 0.1.0-rc1

What's Changed

• bench: harness, contender adapters and shared environment by @krishhgg in #1
• bench: workloads and upstream reference answers by @krishhgg in #2
• bench: stage scripts and analysis tools from the spike and speed rounds by @krishhgg in #3
• docs: research reports on models, runtimes, Rust and Apple silicon by @krishhgg in #4
• docs: bake-off report, synthesis, spike, server and speed results by @krishhgg in #5
• runtime: vendor mlx-sys 0.6.0 with a build that can link a prebuilt MLX by @krishhgg in #6
• laya-core: model config, weights and checkpoint resolution by @krishhgg in #7
• laya-core: question parsing and tokenizer by @krishhgg in #8
• laya-core: backend trait and sequence builder by @krishhgg in #9
• laya-core: answer decoding and the action head by @krishhgg in #10
• laya-core: Agent, test backend and fixture parity tests by @krishhgg in #11
• laya-mlx: the MLX backend by @krishhgg in #12
• sys1-bench: bench adapter and probe by @krishhgg in #13
• sys1d: configuration and request validation by @krishhgg in #14
• sys1d: inference worker and HTTP server by @krishhgg in #15
• sys1d: live end-to-end test against the real model by @krishhgg in #16
• laya-mlx: dense local attention, head pruning and unpadding (off by default) by @krishhgg in #17
• sys1d: make the speed settings the default by @krishhgg in #18
• docs: README with status, supported models, build and run by @krishhgg in #19
• Rewrite the README with diagrams by @krishhgg in #20
• README: call 17 ms inference time; 401 only when a key is set by @krishhgg in #21
• Greptile: review rules for custom Metal kernels and mlx-sys calls by @krishhgg in #22
• laya-mlx: wrapper for MLX custom Metal kernels through mlx-sys by @krishhgg in #23
• laya-mlx: fuserope, one kernel for the qkv split, RoPE and unpad expand by @krishhgg in #24
• sys1d: fuserope in the default settings, round 2 of SPEED.md by @krishhgg in #25
• README: speed numbers from round 2 by @krishhgg in #26
• REPORT.md: the powermetrics sudo rule is removed by @krishhgg in #27
• laya-mlx: banded local attention (band=N), off by default by @krishhgg in #28
• laya-mlx: NAX gemm kernels (nax=all), off by default by @krishhgg in #29
• laya-mlx: fast loading settings (directload, sharehead, parallel_load), off by default by @krishhgg in #30
• sys1d: round 3 defaults (band=512, nax=all, loading settings, MLX_MAX_MB_PER_BUFFER=10) by @krishhgg in #31
• README: speed numbers from round 3 by @krishhgg in #32
• README: open with the speed chart by @krishhgg in #33
• Rename sys1d to sys1rust and add serve and models subcommands by @krishhgg in #34
• Download Laya models into the Hugging Face cache by @krishhgg in #35
• Release bundles, smoke test, Homebrew formula and release workflow by @krishhgg in #36
• README: install with Homebrew by @krishhgg in #37
• Install sys1rust from GitHub Releases and remove Homebrew by @krishhgg in #38
• Guide coding agents through sys1rust setup and use by @krishhgg in #39

New Contributors

• @krishhgg made their first contribution in #1

Full Changelog: https://github.com/krishhgg/sys1rust/commits/v0.1.0-rc1