system1-omni v0.1.0

System1-Omni v0.1.0: Faster Decision Inference

System1-Omni's first release brings native Rust/CUDA execution to decision-model serving, with speed gains backed by reproducible comparisons.

Speed highlights

• Open-Jev: 7.47× faster than raw Hugging Face Transformers. Mean warm HTTP latency dropped from 362.21 ms to 48.50 ms on one H200, using BF16, concurrency 1 and 74 single-candidate JevBench noul requests per pass. Benchmark
• LAYA: 24.9%–37.6% lower warm CLI latency. Optimized RoPE/GEMM configurations reduced p50 across five batch-1 inputs in a matched native-engine comparison on one H800. HTTP, startup and cold Graph capture were excluded. Benchmark

These historical measurements cover specific workloads and configurations; the release commit has not been freshly benchmarked. See the benchmark details below for controls and validation limits.

What's included

Models return structured decisions from a prefill pass. This release supports text decisions with LAYA, Cua-S1 and Open-Jev, plus screenshot-conditioned choices through the Cua-S1 Python worker.

Native model serving

• LAYA English on Hopper: native Rust/CUDA inference for choice, score and noul, including request preprocessing, checkpoint validation, encoder and decision layers, GPU scorer/action heads, and response decoding. Reusable workspaces, shape-specific GEMMs, optimized RoPE and encoder/decision CUDA Graphs target batch-1 latency. The worker warms up before serving and shares the native admission runtime. #21, #90
• Cua-S1 4B 0.2 text: native Rust/CUDA execution of the text adapter, alongside a Transformers/PEFT correctness-reference worker. Supports choice questions with up to 26 options and opt-in exact-length CUDA Graph replay. #13, #19, #52
• Open-Jev-27B-v1.1: native Rust/CUDA text serving with merged LoRA weights, independent candidate prompts, a trained scalar head and saved calibration temperature. Supports choice, score and noul; shares the Qwen prefill implementation with Cua-S1. Includes fused attention gating, residual RMSNorm and packed BF16 SiLU optimizations, Gated DeltaNet preparation improvements, and opt-in CUDA Graph replay. #55, #68

CPU, Apple Silicon and screenshot workers

• LAYA CPU and Apple Silicon: an upstream CPU-worker recipe and an in-repository Python/PyTorch MPS worker, with real inference before readiness, live device reporting, optional compilation and FP16 weights on MPS. #30, #67
• Cua-S1 screenshots: a Python CUDA worker for one PNG/JPEG screenshot and choice questions. Multi-question requests reuse image preprocessing and vision features and restrict the reused-path output projection to the final token. #12, #17, #18
• Experimental screenshot Graph modes: opt-in exact-length, rule-bucket and automatic selection, with capture admission, cooldown, shared resource budgets and eager fallback. Eager remains the default; Graph modes can regress on unsuitable workloads. #22, #33, #37, #38

Serving, runtime and tooling

• Rust HTTP frontend: POST /v1/systemone and GET /health, forwarding to a separately running worker while preserving response status, content type and body. Includes connection reuse, separate health timeouts, bounded buffered responses and expanded transport tests. #2, #42, #58
• Shared native runtime: separate processing and execution, plus FIFO admission and blocking dispatch per loaded executor. Queued cancellation prevents dispatch; already-dispatched work retains its permit and resources until completion. #78, #80
• Reproducible benchmarking: HTTP request replay, latency/throughput/error reports, saved responses and provenance, output-fidelity comparisons, and probability validation that accounts for wire rounding. #40, #75
• Documentation and onboarding: a documentation site, English and Chinese CPU walkthroughs, a recorded first-decision demo, model/hardware support matrix, model recipes and contribution guidance. #51, #60, #89

Benchmark details

• Open-Jev: two measured passes per backend, comparing native eager execution with raw HF using unmerged LoRA and PyTorch fallback operations. Native and HF agreed on all 74 thresholded decisions, but probabilities were not identical. The result measures the complete backend change; it does not isolate individual kernels or establish a general advantage over OpenJev-Fast.
• LAYA: two 100-sample passes per input, using a frozen CLI on a shared H800 with unlocked clocks. Both configurations used Graph replay and reusable workspaces. Timing includes parsing, tokenization, uploads, inference, readback and serialization. The integrated HTTP worker has not been retimed.

Separate kernel, Graph and runtime experiments use different baselines and timing boundaries; their gains must not be added together. Optimization evidence

Get started

Start with Your first decision on CPU (中文): Linux x86_64, Python 3.12, stable Rust, pinned CPU dependencies and the public LAYA English checkpoint. No GPU or weight export is required for this path.

For accelerated execution, follow the LAYA Hopper, Cua-S1 CUDA, Open-Jev CUDA or LAYA Apple Silicon recipe.

Compatibility and current limits

• Native LAYA targets Hopper sm_90a and requires CUDA, TileLang AOT generation and exported rotary tables. It accepts English text only, up to 16 questions, 512 tokens per sequence and 2,048 option markers.
• Native Cua-S1 and Open-Jev require CUDA compute capability 8.0+, a CUDA build and prepared merged checkpoints. Hardware validation is narrower: see the support matrix. Rebuild the shared Qwen CUDA library and both native workers together for ABI 4.
• Native admission is serial per executor. LAYA packs questions within one request; cross-request GPU batching, continuous batching, queue/token budgets and shared processing orchestration remain planned.
• Native Metal, native Cua-S1 screenshot execution, Open-Jev prefix sharing and quantization are not included. Apple GPU support uses Python/PyTorch MPS. Forwarding a payload through the frontend does not imply inference support for that modality.
• The CLM recipe covers stub-encoder contract checks only, not validated real Qwen3-8B decisions. Follow each model's documented validation and numerical limits.

Thanks to everyone who contributed implementations, tests, measurements, documentation and review to this first release.

Source snapshot: 7f39ac4, main as of October 6, 2026; latest included merge: #90 on October 5, 2026. Commit history

What's Changed

• docs: describe architecture and planned LAYA support by @hsliuustc0106 in #4
• Add issue and PR templates for System1-Omni by @hsliuustc0106 in #7
• [Docs] Add stay-tuned star animation to README by @hsliuustc0106 in #5
• docs: scaffold src layout and top-level recipes by @hsliuustc0106 in htt...