Skip to content

BitRun v0.2.0-beta ⚡️

Latest
Latest

Choose a tag to compare

View all tags
@IshekKhal IshekKhal released this 15 Oct 08:43
v0.2.0-beta
0af499c

BitRun v0.2.0-beta ⚡️

BitRun v0.2.0-beta introduces the BitRun Cockpit terminal dashboard, rebases the native inference engine onto modern llama.cpp upstream, eliminates the tensor offset crash on 8B ternary models, and provides verified builds across macOS, Windows 10, and Linux.

Highlights & Key Improvements

  • BitRun Cockpit (bitrun cockpit <model>):

    • Split-pane terminal interface built in Go with Bubble Tea.
    • Left pane features independent scrolling and real-time character-by-character token streaming via server-sent events (/v1/chat/completions).
    • Right pane displays live hardware telemetry (prompt time-to-first-token, generation speed, P-core and E-core utilization) and real-time power metrics (Watts, Joules per token, battery drain estimates).
    • Background server logs are isolated to ~/.bitrun/engine.log with --log-disable to prevent terminal buffer corruption.

  • Rebased Modern Engine (bitrun-engine):

    • Replaced the frozen bitnet.cpp research submodule with a sovereign branch of mainline ggerganov/llama.cpp (b1395).
    • Dynamic GGUF Type 42 remapping in gguf.cpp (GGML_TYPE_PQ2_0 = 142), allowing bonsai-8b to load and generate without memory corruption.
    • Packed 2-bit I2_S memory safety: global .blck_size = 4 stride calculation and 32-byte scale tail preservation to prevent forward mul_mat segmentation faults.
    • Scott Guyton's ARM NEON dot-product kernel (vdotq_s32) and squared ReLU activation (LLM_FFN_RELU_SQR).
    • Architecture fallback macro definitions in arch-fallback.h resolving x86_64 external symbols on MSVC and GCC.

  • Complete 6-Model Registry Catalog:

    • bitnet-70m (79 MB, Q8_0)
    • falcon3-1b (1.30 GB, packed 2-bit I2_S)
    • bitnet-2b (1.13 GB, packed 2-bit I2_S)
    • falcon3-3b (2.07 GB, packed 2-bit I2_S)
    • bonsai-1.7b (237 MB, PrismML Group-128 Q1_0)
    • bonsai-8b (2.08 GB, PrismML Group-128 PQ2_0)

Cross-Platform Hardware Benchmarks (--tokens 64)

All figures below are measured on physical hardware running native binaries with pure CPU SIMD execution (-ngl 0). Zero GPU offloading.

1. Token Processing & Generation Speed (tokens/sec)

Model Size macOS (M4 Pro) Prompt / Gen Windows 10 (i5-12600K) Prompt / Gen Linux WSL2 (i5-12600K) Prompt / Gen
bitnet-70m 79 MB 11,549.2 / 1,619.8 t/s 1,766.2 / 597.8 t/s 333.1 / 134.1 t/s
bonsai-1.7b 237 MB 411.7 / 72.8 t/s 152.4 / 85.6 t/s 41.8 / 16.8 t/s
falcon3-1b 1.30 GB 170.8 / 73.2 t/s 14.9 / 11.1 t/s 16.7 / 6.7 t/s
bitnet-2b 1.13 GB 94.4 / 31.6 t/s 8.0 / 6.7 t/s 10.9 / 4.4 t/s
falcon3-3b 2.07 GB 80.6 / 33.1 t/s 7.5 / 6.0 t/s 8.7 / 3.5 t/s
bonsai-8b 2.08 GB 30.1 / 21.0 t/s 4.7 / 4.0 t/s 1.3 / 0.5 t/s

2. Power Consumption & Energy Efficiency (Watts and Joules/token)

Model Size macOS (M4 Pro) Power / Energy Windows 10 (i5-12600K) Power / Energy Linux WSL2 (i5-12600K) Power / Energy
bitnet-70m 79 MB 11.7 W / 0.043 J/tok 22.6 W / 0.038 J/tok 8.5 W / 0.063 J/tok
bonsai-1.7b 237 MB 11.7 W / 0.316 J/tok 22.6 W / 0.264 J/tok 8.5 W / 0.505 J/tok
falcon3-1b 1.30 GB 11.7 W / 0.345 J/tok 22.6 W / 2.036 J/tok 8.5 W / 1.261 J/tok
bitnet-2b 1.13 GB 11.7 W / 0.484 J/tok 22.6 W / 3.373 J/tok 8.5 W / 1.938 J/tok
falcon3-3b 2.07 GB 11.7 W / 0.592 J/tok 22.6 W / 3.766 J/tok 8.5 W / 2.439 J/tok
bonsai-8b 2.08 GB 11.7 W / 0.886 J/tok 22.6 W / 5.650 J/tok 8.5 W / 16.058 J/tok

One-Line Installers

macOS & Linux:

curl -fsSL https://raw.githubusercontent.com/IshekKhal/bitrun/main/install.sh | sh

Windows (PowerShell):

irm https://raw.githubusercontent.com/IshekKhal/bitrun/main/install.ps1 | iex
Assets 5
Loading