watchOS Benchmark: WhisperKit tiny.en PASS on Apple Watch Series 11 (19.2s load, 4.2s transcribe, +43MB)

[jj33]

WhisperKit on watchOS — Apple Watch Series 11 Benchmark

Submitted for community reference. Raw spike data from a production EMS app feasibility study.

---

Summary

We ran a minimal feasibility spike to determine whether WhisperKit tiny.en can run on Apple Watch Series 11 before committing to a Watch-local offline transcription feature. The result was a clean PASS.

---

Hardware & Software

Device	Apple Watch Series 11 (GPS)
watchOS	11.x
WhisperKit	0.9.x
Model	openai_whisper-tiny.en (73 MB CoreML + tokenizer)
Xcode	16.x

---

Results

Metric	Result	Pass Criteria
Model load time	19.2s	< 30s
Transcription latency	4.2s	< 10s
Memory before load	44 MB	—
Memory after load + transcribe	87 MB	—
Memory delta	+43 MB	Within foreground limit
Process killed (jetsam)	No	Must not be killed
Transcript (5s silent audio)	[BLANK_AUDIO]	Correct

Verdict: PASS — no hardware restriction. Series 11 can run WhisperKit tiny.en to completion.

---

Method

Standalone watchOS spike app. Model files (CoreML .mlmodelc + tokenizer files: tokenizer.json, tokenizer_config.json, vocab.json, merges.txt) bundled directly in the Watch app — Watch cannot reach HuggingFace Hub at runtime.

let config = WhisperKitConfig(modelFolder: modelFolder.path, verbose: false, logLevel: .none)
let whisper = try await WhisperKit(config)
let silentSamples = [Float](repeating: 0.0, count: 16000 * 5)
let results = try await whisper.transcribe(audioArray: silentSamples)

Memory measured via mach_task_basic_info / resident_size before and after.

---

Key Finding: Tokenizer Must Be Bundled

The argmaxinc/whisperkit-coreml model download does not include tokenizer files. watchOS cannot reach HuggingFace Hub at runtime, so loadTokenizerIfNeeded() fails with offlineModeError("Repository not available locally") unless you also bundle the tokenizer files from openai/whisper-tiny.en.

Required files alongside the .mlmodelc folders:

• tokenizer.json
• tokenizer_config.json
• vocab.json
• merges.txt

WhisperKit's loadTokenizer() searches additionalSearchPaths including modelFolder for tokenizer.json — placing all files in the same folder as the .mlmodelc files is sufficient.

---

Deployment Note

Direct Watch deployment in Xcode thrashes on Series 11 (repeated "Connecting / Waiting to reconnect" loop). Reliable workaround: start an active workout on the Watch before deploying. This keeps the screen on and prevents aggressive OS sleep/lock that breaks the Xcode debug tunnel.

---

Design Implication

19.2s load time is not on-demand viable. The model must be pre-warmed at session/call start, not triggered at first voice attempt. For our use case (EMS call logging), we load on call start and the model is ready by the time the medic first speaks.

---

Context

MedicLog is an EMS medication logging app for Apple Watch + iPhone. We're building Watch-local transcription as a fallback when the phone is unreachable (ambulance RF interference, BT drop). This spike confirmed the hardware is capable; production implementation is underway.

---

Spike methodology available on request. Happy to share the full test harness if useful for the WhisperKit watchOS documentation.

[ZachNagengast]

Cool! Thanks for the writeup 👍