Skip to content

TurboFieldfare v0.1 — Initial Public Release

Choose a tag to compare

View all tags
@drumih drumih released this 20 Jul 22:09
v0.1
2763302
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

TurboFieldfare is a custom Swift and Metal inference runtime for running the
instruction-tuned Gemma 4 26B-A4B model on Apple Silicon Macs, including
machines with 8 GB of RAM.

Instead of loading the entire 14.3 GB model into memory, TurboFieldfare keeps
the shared model core and KV cache resident while streaming routed experts from
SSD as they are needed. A typical runtime footprint is about 2 GB.

Highlights

  • Runs Gemma 4 26B-A4B IT on Apple Silicon Macs with 8 GB of RAM
  • Custom Swift 6.2 and Metal 4 inference implementation
  • Native macOS app for model installation and generation
  • Command-line interface for instruction chat and raw completion
  • Streaming installer that downloads and repacks the pinned model without
    materializing the complete checkpoint
  • 4-bit affine embedding, attention, shared-expert, and routed-expert weights
  • SSD-backed expert streaming with a bounded in-memory cache
  • Chunked prompt prefill and token-by-token generation
  • Verified Gemma chat formatting with optional system messages
  • FP16 KV cache with bounded storage for sliding-window attention
  • Apache 2.0 source release

Measured performance

Hardware Decode rate Reported memory
8 GB M2 MacBook Air 5.10–6.30 tok/s about 1.9–2.1 GB
24 GB M5 Pro 31–35 tok/s about 2.1 GB

These are workload-specific measurements, not performance limits. Prompt
length, generated length, cache state, and hardware affect throughput.

See the benchmark methodology and complete results.

Getting started

Requirements:

  • Apple Silicon Mac
  • macOS 26 or later
  • Xcode 26
  • Swift 6.2 or later
  • Approximately 15 GB of free storage for the model
git clone https://github.com/drumih/turbo-fieldfare.git
cd turbo-fieldfare
swift build -c release
.build/release/TurboFieldfareMac

When the app opens, choose Install. TurboFieldfare will download the pinned
model from Hugging Face and repack it into its streaming .gturbo format.
Model weights are not included in this release.

Included products

  • TurboFieldfare — Swift inference library and Metal kernels
  • TurboFieldfareMac — native macOS application
  • TurboFieldfareDecodeService — local model-owning service used by the app
  • TurboFieldfareCLI — instruction-chat and raw-completion CLI
  • TurboFieldfareRepack — streaming model installer and verifier

Scope and limitations

TurboFieldfare is a source-only experimental research release. It currently
supports text-only inference with the pinned Gemma 4 26B-A4B instruction
checkpoint.

Tool calling, images, audio, and operating systems older than macOS 26 are not
supported. Generated output can be incorrect or repetitive, so important
results should be checked.

The model weights remain governed by their original terms and are downloaded
separately. TurboFieldfare is an independent project and is not affiliated
with, sponsored by, or endorsed by Google.

Read the system design
or explore the optimization journey
for implementation details and measured experiments.

Assets 2
Loading