Skip to content

TurboFieldfare 0.2 — OpenAI-Compatible Local Server

Choose a tag to compare

View all tags
@drumih drumih released this 27 Jul 23:25
0.2
e65da9e
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

TurboFieldfare 0.2 adds an experimental OpenAI-compatible server for using
Gemma 4 26B-A4B with local API clients on Apple Silicon.

OpenAI-compatible local server

  • Serves GET /health, GET /v1/models, and POST /v1/chat/completions.
  • Returns standard JSON responses or streams tokens with Server-Sent Events.
  • Accepts system, developer, user, assistant, and tool messages for multi-turn
    conversations.
  • Reuses one verified KV prefix for compatible conversation continuations and
    reports reused tokens through usage.prompt_tokens_details.cached_tokens.
  • Returns OpenAI-style function calls for the client to authorize and execute.
  • Queues requests with bounded memory, recovers cleanly from cancellation, and
    handles SIGINT and SIGTERM with graceful teardown.

The server binds only to 127.0.0.1. It has no authentication or TLS, so do
not expose it through a proxy or tunnel. Tool execution remains under the
client's permission policy.

Start the server

swift build -c release --product TurboFieldfareServer
.build/release/TurboFieldfareServer \
  --model scratch/gemma4.gturbo \
  --port 8080 \
  --max-context 16384

Connect OpenAI-compatible clients to http://127.0.0.1:8080/v1. See the
server guide
for OpenAI Python and OpenCode examples, prompt caching, tool loops, and the
supported request fields.

Current scope

The server supports one local model, one choice, and the Chat Completions API.
It does not support the Responses API, legacy Completions, embeddings,
multimodal input, structured output, batching, log probabilities, or remote
model switching. The Mac app UI remains single-turn.

Full changelog: 0.1.1...0.2

Implemented in #13.

Assets 2
Loading