Skip to content

Releases: torinriley/alpaca

Release list

alpaca.swift 0.1.0

alpaca.swift 0.1.0 Pre-release
Pre-release

Choose a tag to compare

View all tags
@torinriley torinriley released this 09 Oct 17:14
v0.1.0
e89affe

alpaca.swift 0.1.0 (pre-release)

Author: Torin Etheridge · 2026-10-09 · MIT License

A from-scratch Swift and Metal inference runtime for Llama-family language models on Apple Silicon. No llama.cpp, MLX, Core ML, ONNX or Python at runtime.

Highlights

  • Fast. On SmolLM2-135M-Instruct (Apple M5), against llama.cpp b11429 on the same files in the same run: decode is 1.02–1.38× faster at 16–4096 tokens of context; prefill is at parity at 128 tokens (Q8_0: 17,351 vs 17,163 tok/s) and 0.83–0.93× at 512–4096 tokens on quantised files. Compared with the first implementation, prefill is 3× (128 tokens) to 15× (4096 tokens) faster and long-context decode 2–3.5× faster.
  • Validated. CPU backend matches Hugging Face float32 to 5e-5 (F16) / 7e-5 (Q8_0) on logits; 191/191 tokenizer strings produce identical token IDs; 72/72 greedy tokens match on F16, Q8_0 and Q4_0; every reduced-precision GPU component has its own analytically bounded test.
  • Robust. Bounds-checked GGUF parser, survived 20,000 mutation-fuzzed files.
  • Idiomatic API. async loading, AsyncSequence streaming, cancellation, seeded sampling, isolated sessions, memory-budget check; alpaca CLI (info, generate, bench, profile) and a SwiftUI example app.

Supported

macOS 14+ / iOS 17+ on Apple Silicon · Swift 6 · Llama-style decoders · GGUF v2/v3 · F16, Q8_0, Q4_0 (Q4_1 expanded to F16 at load) · byte-level BPE tokenizers (gpt2, smollm).

Known limitations

  • Validated on one machine (Apple M5) and one model. Not run on a physical iPhone or iPad; the library and example app compile for the iOS Simulator only.
  • The fast prefill GEMM needs an Apple10-family GPU (M5/A19) and macOS/iOS 26; elsewhere the exact float32 path runs at roughly half the prefill throughput (measured on M5 only).
  • The GPU-resident decode loop is greedy-only; stochastic sampling is ~15–20% slower.
  • No RoPE scaling, SentencePiece tokenizers, K-quants, MoE or continuous batching.
  • Performance numbers were taken on a machine with background load; compare within a run. See Docs/PERFORMANCE.md.

Install

.package(url: "https://github.com/torinriley/alpaca.git", from: "0.1.0")

Documentation: README.md, Docs/ARCHITECTURE.md, Docs/NUMERICAL_VALIDATION.md, Docs/PERFORMANCE.md, Docs/MODEL_SUPPORT.md, Docs/DEVELOPMENT.md.

Loading