system1-omni v0.1.0

System1-Omni v0.1.0: Faster Decision Inference

System1-Omni's first release brings native Rust/CUDA execution to decision-model serving, with speed gains backed by reproducible comparisons.

Speed highlights

• Open-Jev: 7.47× faster than raw Hugging Face Transformers. Mean warm HTTP latency dropped from 362.21 ms to 48.50 ms on one H200, using BF16, concurrency 1 and 74 single-candidate JevBench noul requests per pass. Benchmark
• LAYA: 24.9%–37.6% lower warm CLI latency. Optimized RoPE/GEMM configurations reduced p50 across five batch-1 inputs in a matched native-engine comparison on one H800. HTTP, startup and cold Graph capture were excluded. Benchmark

These historical measurements cover specific workloads and configurations; the release commit has not been freshly benchmarked. See the benchmark details below for controls and validation limits.

What's included

Models return structured decisions from a prefill pass. This release supports text decisions with LAYA, Cua-S1 and Open-Jev, plus screenshot-conditioned choices through the Cua-S1 Python worker.

Native model serving

• LAYA English on Hopper: native Rust/CUDA inference for choice, score and noul, including request preprocessing, checkpoint validation, encoder and decision layers, GPU scorer/action heads, and response decoding. Reusable workspaces, shape-specific GEMMs, optimized RoPE and encoder/decision CUDA Graphs target batch-1 latency. The worker warms up before serving and shares the native admission runtime. #21, #90
• Cua-S1 4B 0.2 text: native Rust/CUDA execution of the text adapter, alongside a Transformers/PEFT correctness-reference worker. Supports choice questions with up to 26 options and opt-in exact-length CUDA Graph replay. #13, #19, #52
• Open-Jev-27B-v1.1: native Rust/CUDA text serving with merged LoRA weights, independent candidate prompts, a trained scalar head and saved calibration temperature. Supports choice, score and noul; shares the Qwen prefill implementation with Cua-S1. Includes fused attention gating, residual RMSNorm and packed BF16 SiLU optimizations, Gated DeltaNet preparation improvements, and opt-in CUDA Graph replay. #55, #68

CPU, Apple Silicon and screenshot workers

• LAYA CPU and Apple Silicon: an upstream CPU-worker recipe and an in-repository Python/PyTorch MPS worker, with real inference before readiness, live device reporting, optional compilation and FP16 weights on MPS. #30, #67
• Cua-S1 screenshots: a Python CUDA worker for one PNG/JPEG screenshot and choice questions. Multi-question requests reuse image preprocessing and vision features and restrict the reused-path output projection to the final token. #12, #17, #18
• Experimental screenshot Graph modes: opt-in exact-length, rule-bucket and automatic selection, with capture admission, cooldown, shared resource budgets and eager fallback. Eager remains the default; Graph modes can regress on unsuitable workloads. #22, #33, #37, #38

Serving, runtime and tooling

• Rust HTTP frontend: POST /v1/systemone and GET /health, forwarding to a separately running worker while preserving response status, content type and body. Includes connection reuse, separate health timeouts, bounded buffered responses and expanded transport tests. #2, #42, #58
• Shared native runtime: separate processing and execution, plus FIFO admission and blocking dispatch per loaded executor. Queued cancellation prevents dispatch; already-dispatched work retains its permit and resources until completion. #78, #80
• Reproducible benchmarking: HTTP request replay, latency/throughput/error reports, saved responses and provenance, output-fidelity comparisons, and probability validation that accounts for wire rounding. #40, #75
• Documentation and onboarding: a documentation site, English and Chinese CPU walkthroughs, a recorded first-decision demo, model/hardware support matrix, model recipes and contribution guidance. #51, #60, #89

Benchmark details

• Open-Jev: two measured passes per backend, comparing native eager execution with raw HF using unmerged LoRA and PyTorch fallback operations. Native and HF agreed on all 74 thresholded decisions, but probabilities were not identical. The result measures the complete backend change; it does not isolate individual kernels or establish a general advantage over OpenJev-Fast.
• LAYA: two 100-sample passes per input, using a frozen CLI on a shared H800 with unlocked clocks. Both configurations used Graph replay and reusable workspaces. Timing includes parsing, tokenization, uploads, inference, readback and serialization. The integrated HTTP worker has not been retimed.

Separate kernel, Graph and runtime experiments use different baselines and timing boundaries; their gains must not be added together. Optimization evidence

Get started

Start with Your first decision on CPU (中文): Linux x86_64, Python 3.12, stable Rust, pinned CPU dependencies and the public LAYA English checkpoint. No GPU or weight export is required for this path.

For accelerated execution, follow the LAYA Hopper, Cua-S1 CUDA, Open-Jev CUDA or LAYA Apple Silicon recipe.

Compatibility and current limits

• Native LAYA targets Hopper sm_90a and requires CUDA, TileLang AOT generation and exported rotary tables. It accepts English text only, up to 16 questions, 512 tokens per sequence and 2,048 option markers.
• Native Cua-S1 and Open-Jev require CUDA compute capability 8.0+, a CUDA build and prepared merged checkpoints. Hardware validation is narrower: see the support matrix. Rebuild the shared Qwen CUDA library and both native workers together for ABI 4.
• Native admission is serial per executor. LAYA packs questions within one request; cross-request GPU batching, continuous batching, queue/token budgets and shared processing orchestration remain planned.
• Native Metal, native Cua-S1 screenshot execution, Open-Jev prefix sharing and quantization are not included. Apple GPU support uses Python/PyTorch MPS. Forwarding a payload through the frontend does not imply inference support for that modality.
• The CLM recipe covers stub-encoder contract checks only, not validated real Qwen3-8B decisions. Follow each model's documented validation and numerical limits.

Thanks to everyone who contributed implementations, tests, measurements, documentation and review to this first release.

Source snapshot: 7f39ac4, main as of October 6, 2026; latest included merge: #90 on October 5, 2026. Commit history

What's Changed

• docs: describe architecture and planned LAYA support by @hsliuustc0106 in #4
• Add issue and PR templates for System1-Omni by @hsliuustc0106 in #7
• [Docs] Add stay-tuned star animation to README by @hsliuustc0106 in #5
• docs: scaffold src layout and top-level recipes by @hsliuustc0106 in #8
• Add Rust frontend for Jev workers by @linear3735 in #2
• docs: add Cua-S1 4B 0.2 inference contract by @twu3202 in #11
• docs: add contributor self-review guidance and precheck skill by @hsliuustc0106 in #41
• Add Cua-S1 0.2 multimodal CUDA worker with upstream parity by @Levius-Fubuki in #12
• cua_s1: add a native CUDA text worker by @twu3202 in #19
• cua_s1: add a text worker that loads Qwen3.5-4B through Transformers by @twu3202 in #13
• docs: build a documentation site with MkDocs and deploy it to Pages by @twu3202 in #51
• [Benchmark] Add GPU serving replay and parity harness by @hsliuustc0106 in #40
• cua_s1: add opt-in exact-length CUDA Graph replay by @hsliuustc0106 in #52
• Reuse Cua-S1 image preprocessing and vision features per request by @Levius-Fubuki in #17
• Limit Cua-S1 reused-request output projection to final token by @Levius-Fubuki in #18
• Serve Cua-S1 multimodal requests with bounded segmented CUDA Graphs by @Levius-Fubuki in #22
• Add CPU checkpoint loading for Laya by @linear3735 in #21
• docs: add model contribution skill and placement guidance by @hsliuustc0106 in #65
• docs: add repository-owned independent review skill by @hsliuustc0106 in #62
• docs: add a supported models and hardware page by @twu3202 in #60
• Laya on Apple Silicon: worker, benchmarks and recipe by @cacheline999 in #30
• docs: restructure README with badges, nav links, and section layout by @hsliuustc0106 in #70
• test: expand frontend API integration coverage by @MrDongsls in #58
• laya: follow-ups to #30 — reproducible numbers, corrected recipe claims, wider answer checks by @cacheline999 in #67
• open_jev: add native Rust/CUDA text worker by @hsliuustc0106 in #55
• Add Cua-S1 multimodal reference tensor exports by @Levius-Fubuki in #53
• bench: accept wire-rounded probability sums per label count by @hsliuustc0106 in #75
• docs: align architecture, skills, and integration contracts by @hsliuustc0106 in #76
• qwen3_5: reduce Gated DeltaNet preparation shared memory by @hsliuustc0106 in #68
• refactor: separate native processing from model execution by @hsliuustc0106 in #78
• refactor: share native execution admission and dispatch by @hsliuustc0106 in #80
• docs: add reproducible demo evidence to self-review by @hsliuustc0106 in #79
• [Recipe] Run CLM behind the frontend, and compare engine responses by @xiaoyu-xyz in #23
• docs: add System1-Omni logo to README by @hsliuustc0106 in #81
• Bound Cua-S1 Graph capture work with request admission and cooldown by @Levius-Fubuki in #33
• Add instance-local rule buckets to the Cua-S1 multimodal worker by @Levius-Fubuki in #37
• Add automatic Graph selection with shared Cua-S1 worker budgets by @Levius-Fubuki in #38
• Bound health checks and buffered worker responses by @linear3735 in #42
• docs: make the first CPU decision reproducible by @hsliuustc0106 in #89
• docs: record Open-Jev PR-by-PR optimization A/B results by @hsliuustc0106 in #88
• laya: add native CUDA inference and optimize batch-1 latency by @linear3735 in #90

New Contributors

• @hsliuustc0106 made their first contribution in #4
• @linear3735 made their first contribution in #2
• @twu3202 made their first contribution in #11
• @Levius-Fubuki made their first contribution in #12
• @cacheline999 made their first contribution in #30
• @MrDongsls made their first contribution in #58
• @xiaoyu-xyz made their first contribution in #23

Full Changelog: https://github.com/ThinkFlowLab/system1-omni/commits/release/v0.1.0