FluidUse 0.4.0

Vela 3.0 on Core ML

Vela3Manager runs vela3-fast (Vela 3.0 from vLLM Semantic Router × KR Labs, UltraBERT-380M, Apache-2.0) from FluidInference/vela-3.0-coreml. The snapshot is pinned by revision and checksum, about 1.48 GB.

• Decisions: choice or yes / no questions about a state, decide(state:questions:).
• Spans: 17 types of personal information with pii(_:), and the words of an answer its context does not support with unsupported(context:answer:).
• Long inputs: handled with the release's own rule. The context is cut first, then long text is read in overlapping 1,024-token windows.
• Hardware: one encoder pass per question, on the GPU.
• Vela3Check: compares the Swift path against the Python runtime of the snapshot. Use parity <dir> <fixtures.json> for the comparison, warm <dir> for a cold-start smoke test, and download to fetch the snapshot.

let vela = try await Vela3Manager.load(from: try await Vela3ModelStore.ensure())
let r = try await vela.decide(
    state: "My card was charged twice and I need this fixed today.",
    questions: [.choice(id: "intent", instructions: "What does the customer want?",
                        options: [("refund", "Money back for the duplicate charge"), ("cancel", "Cancel the order")])])
let spans = try await vela.pii("Hi, I'm Dana Whitfield, dana@example.com.").spans

Numbers (Apple M5 Pro, GPU)

Decision, end to end	~10 ms median
PII pass over a paragraph	~18 ms median
vs the Python runtime (same encoders)	0 / 23 token, 0 / 120 decision, 0 / 16 span-set mismatches (29 windows, 599 spans)
Python runtime vs fp32 PyTorch release	0 / 1,200 typed decisions differ; identical PII spans on 25 passages

The Neural Engine is not used for Vela 3. It was slower for this encoder at every length and further from the fp32 answers.

The first GPU prediction on a cold Core ML compile cache sometimes aborted in MPSGraphTensorData on macOS 27. Vela3Manager avoids it by giving Core ML a preallocated output buffer (output backing) with synchronous prediction: 0 aborts in 20 cold runs.

Also since 0.3.0

Decision models

• Decision 3.0 (d3-lite / nano / mini) runtime (#37)
• Decision 2.0 Kai 0.6B / Eos 0.8B: runtime, model store, parity check, issue-triage demo (#25)
• Vela 2.0 0.3B: Swift engine, model store, parity check, guardrail and chatbot front-door demos (#26)
• clef-flash (Cloudflare 9B decision model) and ticket-triage demo (#34); clef-text-0.6b, distilled from clef-flash, with a 1,000-ticket demo (#36); Clef-vision 0.8B (#24)
• d1-omni-600M: runtime, model store, moderation firehose demo (#27)
• Kev-0.8B: fused decisions and Guess Who demo (#19)
• Intern-Decision-0.8B: runtime, model store, Showdown fine-tune snapshot (#21), int8 Showdown snapshot (#22)
• JEV-9B vision and Mario demo with a 9M pixel student (#39)

Embeddings and search

• EmbeddingGemma 2 on the Neural Engine: bookmark sorter and sort-by-topic demo (#29, #30), audio search (#31), images (#32), code search demo (#33)
• Granite-Embedding-30M-Sparse (Evoke): manager, model store, parity check, search demo (#28)
• SigLIP 2 zero-shot image classification and the Sort Photos demo (#18)

Other models and demos

• Unlimited-OCR: multi-page document OCR and PDF reader demo (#38)
• Python writer: Qwen2.5-Coder-0.5B and live demo (#35)
• Short-reply model: Swift host, hotkey demo, parity check (#23)
• Doom demo: SauerkrautLM-Doom on Core ML, and GLiClass on ViZDoom (#20)
• README updates (#14, #41)

Package

.package(url: "https://github.com/FluidInference/FluidUse.git", from: "0.4.0")

Pull requests: #14, #18–#41.