v1.0.4 — Lykuro Native Inference Engine (pre-release)

v1.0.4 (2026-08-15)

GPU decode overhaul — ahead of MLX (Metal) and Ollama (Metal & CUDA)

Metal (Apple Silicon) — greedy generation now runs up to 16 speculative steps per command buffer with on-GPU argmax + embedding gather, one batch always committed ahead of the one being consumed; attention switches to split-row flash decoding (fixed-order merge, deterministic). M4 Pro, 256-token generations, client-side measurement:

Model	This engine (metal-q4)	MLX 4bit (self-reported)	Ollama 0.32.5 q4_K_M
Qwen2.5-0.5B	485 tok/s, TTFT 37 ms	441 tok/s	246 tok/s, 103 ms
Qwen2.5-1.5B	210 tok/s, TTFT 89 ms	214 tok/s	154 tok/s, 101 ms

CUDA (Linux) — vectorized 8-wide weight loads (per-element nibble reads were an 8x DRAM amplification), warp-per-row B=1 GEMV kernels with shared-memory activations, a quantized lm head in the quantized modes, and GreedyRun as back-to-back decode-graph replays with on-GPU argmax. RTX 3060, host-local measurement:

Model	This engine (cuda-q4)	Ollama 0.21.0 q4_K_M
Qwen2.5-0.5B	434 tok/s, TTFT 45 ms	246 tok/s, 116 ms
Qwen2.5-1.5B	183 tok/s, TTFT 140 ms	152 tok/s, 119 ms

Also: cuda-q8[:N] / cuda-q4[:N] backends exposed on the CLI/HTTP surface; incremental detokenization in the generators; new GenerativeModel API SupportsGreedyRun/GreedyRun (temperature > 0 keeps the per-token Decode + CPU sampler path). Parity gates: GreedyRun matches sequential Decode + greedy argmax bit-exactly on both backends' test suites (187 macOS / 188 Linux, all green); outputs stay bit-exact run-to-run.

See docs/RELEASE_NOTES.md for full notes. Same pre-release caveats as v1.0.0 (unsigned macOS binaries pending Developer ID enrollment).

Downloads (SHA-256)

File	Platform / backend	SHA-256
lykuro-native-engine-macos-arm64	macOS Apple Silicon, Metal (single binary, ad-hoc signed)	4741d1fed1cbac97bcbd10f70d45e7e9e8cc5c094978f3a3b31d2e135f68c327
lykuro-native-engine-macos-metal-1.0.4.tar.gz	macOS Apple Silicon, Metal (full package)	b3ed1c47523835db21d0d7db12390b7f1aec8663c921e78e0c09e020cbb97e6b
lykuro-native-engine-linux-cuda-1.0.4.tar.gz	Linux x86_64 + NVIDIA CUDA (full package)	9cfe513d4f9b72d7d507fec5c5fc09614ec7fc3149f1881817d7e4da73d234d6
lykuro-native-engine-linux-amd64	Linux x86_64, CPU (single binary)	02e613a907d0f8daad61d99fe4b7e1fb9c6f52156508e3ac2015e9d4a42938ec
lykuro-native-engine-linux-arm64	Linux aarch64, CPU (single binary)	10586713501e6ad3a33eae45ea7a6473db7871b7d6f94583f4665a29cea39703

Runtime requirements: Linux binaries need glibc + system OpenSSL (CUDA package additionally needs the NVIDIA driver/CUDA runtime); macOS binaries need Apple Silicon (arm64) and are ad-hoc signed — browser downloads need xattr -d com.apple.quarantine once.