BitRun v0.1.0-alpha — The 1-bit LLM Runner

BitRun v0.1.0-alpha ⚡️

The single-binary, zero-compile runner and model manager for Microsoft BitNet b1.58 and 1.58-bit ternary LLMs on commodity CPUs.

Key Highlights

• Zero-Compile & Single-Binary: No Python, CMake, Clang, or Docker needed.
• Pure CPU SIMD Inference: Extreme speed on commodity CPUs using ARM NEON DotProd and x86 AVX2.
• Model Catalog: Automatic streaming downloads for falcon3-1b, bitnet-2b, and bitnet-70m with terminal progress bar.
• Custom Personas (Modelfiles): Create custom system prompts and parameter presets via bitrun create <name> -f Modelfile.
• OpenAI-Compatible REST Server: Local endpoint on port 11435 compatible with Cursor, Continue.dev, and OpenAI SDKs.

Verified Benchmarks (Pure CPU)

• Apple Silicon M4 Pro: 137.5 t/s prompt | 98.6 t/s generation (falcon3-1b), 1,355 t/s generation (bitnet-70m).
• Intel Core i5-12600K (AVX2): 93–210 t/s prompt | 40–45 t/s generation (falcon3-1b), 492 t/s generation (bitnet-70m).
• Linux x86_64 (WSL2 / Ubuntu): ~130 ms TTFT | 40–45 t/s generation (falcon3-1b), < 25 ms TTFT (bitnet-70m).

One-Line Installation

macOS (Apple Silicon) & Linux (x86_64):

curl -fsSL https://raw.githubusercontent.com/IshekKhal/bitrun/main/install.sh | sh

Windows (PowerShell):

irm https://raw.githubusercontent.com/IshekKhal/bitrun/main/install.ps1 | iex

Upstream Contributions

• microsoft/BitNet#642: ARM64 2B batch stride, FFN activation, and tokenizer inference fix.
• microsoft/BitNet#644: Native MSVC compiler support and CLI build tools on Windows.

Open Source Credits

Built with gratitude on top of:

• llama.cpp (MIT License) by Georgi Gerganov and the open-source community.
• BitNet.cpp (MIT License) by Microsoft Research.