TurboFieldfare 0.3 — Apple10 Long-Context Prefill

TurboFieldfare 0.3 speeds up long-context prefill on Apple10 GPUs and fixes the Mac app's download ETA countdown.

Apple10 long-context prefill

• Full-attention layers now process all eight GQA query heads together with TensorOps, reusing K/V reads.
• The new path is selected automatically on Apple10 for the pinned full-attention shape. Earlier GPUs keep causal-tiled attention.
• Isolated full attention measured 11.24x faster at 16K and 11.63x faster at 64K.
• On the same 32K input, full-runtime prefill fell from 491.09 to 204.29 seconds, a 2.404x speedup without increased memory use.
• Direct attention checks passed through 64K, and MLX-relative endpoints matched top-1 at 8K, 16K, 32K, and 64K.

Mac app

• The visible download ETA now updates after each estimation interval, so a steady download advances the minute countdown.

Validation

• The release build, serial package tests, and Markdown link check passed on the release commit.
• Thirteen Apple10 tile-boundary cases passed against the CPU reference and were byte-stable across repeated runs on an M5 Pro.

This is a source-only release. It does not include model weights or application binaries.

Full changelog: 0.2.1...0.3

Implemented in #16 and #17.

TurboFieldfare 0.2.1 — Resumable Model Downloads

TurboFieldfare 0.2.1 makes the model download resumable in the Mac app and
command-line installer. Completed ranges survive cancellation or process
interruption and are revalidated before reuse.

Resumable model downloads

• The Mac app can resume or discard a saved download and reports reused bytes,
  current progress, and estimated time remaining.
• The installer adds --resume and --discard-partial commands.
• A compact checkpoint binds saved ranges to the source revision and copy plan.
• Resume verifies saved destination digests and downloads only missing or
  damaged ranges.
• Exact HTTP range validation, bounded temporary storage, retries, a
  physical-target lock, final verification, and atomic publication protect the
  installation.

Continue from the command line

swift run -c release TurboFieldfareRepack \
  --output scratch/gemma4.gturbo \
  --overwrite \
  --resume

To remove a saved download:

swift run -c release TurboFieldfareRepack \
  --discard-partial \
  --output scratch/gemma4.gturbo

Validation

• A release installer killed after three durable ranges resumed with
  200,228,864 bytes reused and completed all 223 planned ranges with zero
  retries.
• Final verification passed 37 files and 14,291,915,755 bytes.
• The public serial test suite passed 509 tests in 108 suites.

This is a source-only release. It does not include model weights or application
binaries.

Full changelog: 0.2...0.2.1

Implemented in #14.

TurboFieldfare 0.2 — OpenAI-Compatible Local Server

TurboFieldfare 0.2 adds an experimental OpenAI-compatible server for using
Gemma 4 26B-A4B with local API clients on Apple Silicon.

OpenAI-compatible local server

• Serves GET /health, GET /v1/models, and POST /v1/chat/completions.
• Returns standard JSON responses or streams tokens with Server-Sent Events.
• Accepts system, developer, user, assistant, and tool messages for multi-turn
  conversations.
• Reuses one verified KV prefix for compatible conversation continuations and
  reports reused tokens through usage.prompt_tokens_details.cached_tokens.
• Returns OpenAI-style function calls for the client to authorize and execute.
• Queues requests with bounded memory, recovers cleanly from cancellation, and
  handles SIGINT and SIGTERM with graceful teardown.

The server binds only to 127.0.0.1. It has no authentication or TLS, so do
not expose it through a proxy or tunnel. Tool execution remains under the
client's permission policy.

Start the server

swift build -c release --product TurboFieldfareServer
.build/release/TurboFieldfareServer \
  --model scratch/gemma4.gturbo \
  --port 8080 \
  --max-context 16384

Connect OpenAI-compatible clients to http://127.0.0.1:8080/v1. See the
server guide
for OpenAI Python and OpenCode examples, prompt caching, tool loops, and the
supported request fields.

Current scope

The server supports one local model, one choice, and the Chat Completions API.
It does not support the Responses API, legacy Completions, embeddings,
multimodal input, structured output, batching, log probabilities, or remote
model switching. The Mac app UI remains single-turn.

Full changelog: 0.1.1...0.2

Implemented in #13.

TurboFieldfare 0.1.1

What's Changed

• Rewrite the optimization journey by @drumih in #4
• Make the system design easier to follow by @drumih in #5

Full Changelog: v0.1...0.1.1

TurboFieldfare v0.1 — Initial Public Release

TurboFieldfare is a custom Swift and Metal inference runtime for running the
instruction-tuned Gemma 4 26B-A4B model on Apple Silicon Macs, including
machines with 8 GB of RAM.

Instead of loading the entire 14.3 GB model into memory, TurboFieldfare keeps
the shared model core and KV cache resident while streaming routed experts from
SSD as they are needed. A typical runtime footprint is about 2 GB.

Highlights

• Runs Gemma 4 26B-A4B IT on Apple Silicon Macs with 8 GB of RAM
• Custom Swift 6.2 and Metal 4 inference implementation
• Native macOS app for model installation and generation
• Command-line interface for instruction chat and raw completion
• Streaming installer that downloads and repacks the pinned model without
  materializing the complete checkpoint
• 4-bit affine embedding, attention, shared-expert, and routed-expert weights
• SSD-backed expert streaming with a bounded in-memory cache
• Chunked prompt prefill and token-by-token generation
• Verified Gemma chat formatting with optional system messages
• FP16 KV cache with bounded storage for sliding-window attention
• Apache 2.0 source release

Measured performance

Hardware	Decode rate	Reported memory
8 GB M2 MacBook Air	5.10–6.30 tok/s	about 1.9–2.1 GB
24 GB M5 Pro	31–35 tok/s	about 2.1 GB

These are workload-specific measurements, not performance limits. Prompt
length, generated length, cache state, and hardware affect throughput.

See the benchmark methodology and complete results.

Getting started

Requirements:

• Apple Silicon Mac
• macOS 26 or later
• Xcode 26
• Swift 6.2 or later
• Approximately 15 GB of free storage for the model

git clone https://github.com/drumih/turbo-fieldfare.git
cd turbo-fieldfare
swift build -c release
.build/release/TurboFieldfareMac

When the app opens, choose Install. TurboFieldfare will download the pinned
model from Hugging Face and repack it into its streaming .gturbo format.
Model weights are not included in this release.

Included products

• TurboFieldfare — Swift inference library and Metal kernels
• TurboFieldfareMac — native macOS application
• TurboFieldfareDecodeService — local model-owning service used by the app
• TurboFieldfareCLI — instruction-chat and raw-completion CLI
• TurboFieldfareRepack — streaming model installer and verifier

Scope and limitations

TurboFieldfare is a source-only experimental research release. It currently
supports text-only inference with the pinned Gemma 4 26B-A4B instruction
checkpoint.

Tool calling, images, audio, and operating systems older than macOS 26 are not
supported. Generated output can be incorrect or repetitive, so important
results should be checked.

The model weights remain governed by their original terms and are downloaded
separately. TurboFieldfare is an independent project and is not affiliated
with, sponsored by, or endorsed by Google.

Read the system design
or explore the optimization journey
for implementation details and measured experiments.