Swift Speech Transcriber

Transcribe audio and video to timestamped text entirely on-device. No network access, no third-party dependencies, no API keys, no per-minute cost — pure Apple SpeechAnalyzer.

Built on Apple's SpeechAnalyzer / SpeechTranscriber (macOS 26 / iOS 26). Accepts a local audio file, a remote audio URL, or a local/remote video URL, and returns a structured transcript with sentence-level timestamps and SRT/VTT/JSON export.

Features

• 🎯 Simple API — transcribe with a single async call
• 🎙️ On-device — Apple SpeechAnalyzer, completely free, no data leaves the machine
• 🧠 Two engines, one API — Apple SpeechAnalyzer (fastest, zero-dependency) or Whisper via WhisperKit (accuracy + 100 languages); both return the same Transcript
• 🔗 Audio or video — local files or remote URLs; video audio tracks are extracted automatically
• 🔴 Live streaming (both engines) — Apple audioInput + mic, or Whisper live mic (VAD-gated); interim + final captions
• 🗣️ Speaker diarization — optional Diarizer product (SpeakerKit / pyannote) labels who spoke when
• 🌐 Translate — Whisper can transcribe-and-translate any language to English in one pass
• 🔤 Custom vocabulary — bias recognition toward names, jargon, and commands the model would otherwise mis-hear (Apple + Whisper); a real help for proper nouns and strong accents
• ⏱️ Timestamped segments — sentence-level (or word-level) timing for every line
• 💬 Subtitle export — SubRip (.srt) and WebVTT (.vtt) out of the box
• 📦 JSON export — snake_case JSON, great for LLM input or storage
• 🌍 Locale management — discover supported/installed languages and pre-download models
• 🎤 Microphone helper — built-in MicrophoneCapture for live transcription
• 📊 Progress reporting — real-time callbacks for UI integration
• 🔒 Sandbox-friendly — temp files only, no security-scoped grants required
• 🍎 Cross-platform — macOS 26+, iOS 26+
• 🔒 Zero dependencies (Apple engine) — the SpeechTranscriber product uses only Speech + AVFoundation; WhisperKit is an opt-in second product

Requirements

• macOS 26.0+ / iOS 26.0+
• Swift 6.2+
• Xcode 26.0+

Installation

Swift Package Manager

Add the following to your Package.swift:

dependencies: [
    .package(url: "https://github.com/arraypress/swift-speech-transcriber.git", from: "1.0.0")
]

The package vends three library products — depend on whichever you need:

• SpeechTranscriber — the Apple SpeechAnalyzer engine. Zero dependencies.
• WhisperTranscriber — the Whisper engine (pulls in WhisperKit). Returns the same Transcript type.
• Diarizer — speaker diarization (pulls in SpeakerKit). Labels an existing Transcript with who spoke.

Usage

Basic Transcription

import SpeechTranscriber

// A local audio file
let transcript = try await SpeechTranscriber.transcribe(audioURL: fileURL)

// Plain text dump (great for LLM input)
print(transcript.plainText)

// Timestamped, one segment per line
for segment in transcript.segments {
    print("[\(segment.formattedStart)] \(segment.text)")
}

Whisper Engine (optional)

Same call shape, same Transcript out — swap the engine when you want Whisper's accuracy and 100-language coverage. Models download from Hugging Face on first use and are cached. Use this as the "engine" toggle in your app's settings.

import WhisperTranscriber

let transcript = try await WhisperTranscriber.transcribe(
    audioURL: url,
    model: .largeV3Turbo,   // .auto picks per device; .tiny … .largeV3
    language: nil           // nil = auto-detect, or "en", "de", …
)

print(transcript.plainText)
let srt = transcript.srt    // same export helpers as the Apple engine

A typical settings-driven dispatch:

enum Engine { case apple, whisper }

func transcribe(_ url: URL, using engine: Engine) async throws -> Transcript {
    switch engine {
    case .apple:   return try await SpeechTranscriber.transcribe(audioURL: url)
    case .whisper: return try await WhisperTranscriber.transcribe(audioURL: url, model: .largeV3Turbo)
    }
}

Whisper: Live Microphone

Real-time captions from the mic, Whisper-powered (VAD-gated). Same LiveTranscriptionEvent shape as the Apple engine's live API.

for try await event in WhisperTranscriber.transcribeLive(model: .small) {
    if event.isFinal { transcript += event.text + " " }
    else { liveCaption = event.text }   // updates as you speak
}

Whisper: Translate to English

// Any language in → English transcript out, in one pass
let english = try await WhisperTranscriber.transcribe(audioURL: url, translate: true)

Speaker Diarization

Label an existing transcript (from either engine) with who spoke when.

import Diarizer

let transcript = try await SpeechTranscriber.transcribe(audioURL: url)
let diarized   = try await Diarizer.diarize(transcript, audioURL: url)

print(diarized.speakerCount)   // e.g. 2
print(diarized.plainText)      // "Speaker 1: …\nSpeaker 2: …"

// Or just the raw turns ("who spoke when")
let turns = try await Diarizer.diarize(audioURL: url)

From a Remote URL

// Remote audio — downloaded to a temp file, transcribed, cleaned up
let podcast = try await SpeechTranscriber.transcribe(
    audioURL: URL(string: "https://example.com/episode.mp3")!
)

// A video's audio track — extracted, then transcribed
let clip = try await SpeechTranscriber.transcribe(videoURL: videoURL)

Check Availability

guard SpeechTranscriber.isAvailable else {
    // Present a fallback — this device/OS can't run the on-device engine
    return
}

Custom Options

let options = TranscriptionOptions(
    locale: Locale(identifier: "de-DE"),  // any installed speech locale
    segmentation: .sentence,              // .sentence (default) or .word
    vocabulary: ["arraypress", "Kokoro"]  // bias toward these terms
)
let transcript = try await SpeechTranscriber.transcribe(audioURL: url, options: options)

Custom Vocabulary

Give the recognizer a list of terms — names, brand names, jargon, commands — that it would otherwise mis-hear. This is contextual biasing (making those terms more likely), not a hard constraint, and it's especially helpful for proper nouns and strong accents.

// Apple engine — via TranscriptionOptions
let options = TranscriptionOptions(vocabulary: ["arraypress", "Kokoro", "SwiftUI", "FODMAP"])
let transcript = try await SpeechTranscriber.transcribe(audioURL: url, options: options)

// Whisper engine — via the vocabulary parameter
let transcript = try await WhisperTranscriber.transcribe(
    audioURL: url,
    model: .largeV3Turbo,
    vocabulary: ["arraypress", "Kokoro", "SwiftUI", "FODMAP"]
)

Each engine maps it to its own mechanism: Apple's SpeechAnalyzer contextual strings, and Whisper's decoder prompt (the terms are tokenized and seeded into the prompt). Effectiveness varies by engine — Apple's contextual strings are a firmer bias; Whisper's prompt is softer but also nudges spelling. (Parakeet does not yet support custom vocabulary here.)

Live Streaming

Feed captured audio (microphone, system audio, or a file read in chunks) and receive interim (isFinal == false) and final captions as they're recognised:

let mic = try MicrophoneCapture()

for try await event in SpeechTranscriber.transcribe(audioInput: mic.stream()) {
    if event.isFinal {
        transcript += event.text + " "
    } else {
        liveCaption = event.text   // updates rapidly as you speak
    }
}
mic.stop()

Any source works — just yield AudioInput values into an AsyncStream:

let (stream, continuation) = AsyncStream<AudioInput>.makeStream()
// ...from your own AVAudioEngine tap, ScreenCaptureKit, or file reads...
continuation.yield(AudioInput(pcmBuffer))
continuation.finish()

for try await event in SpeechTranscriber.transcribe(audioInput: stream) { ... }

Microphone access requires NSMicrophoneUsageDescription (and, on macOS, the audio-input entitlement). The OS prompts on first use.

Locale Management

Discover supported and installed languages, and pre-download a model so the first transcription doesn't pay the download cost:

let supported = await SpeechTranscriber.supportedLocales   // [Locale]
let installed = await SpeechTranscriber.installedLocales

if await SpeechTranscriber.isSupported(locale),
   await !SpeechTranscriber.isInstalled(locale) {
    try await SpeechTranscriber.prepare(locale: locale) { progress in
        print(progress.statusText)
    }
}

Subtitle Export

let transcript = try await SpeechTranscriber.transcribe(videoURL: videoURL)

try transcript.srt.write(to: srtURL, atomically: true, encoding: .utf8)
try transcript.vtt.write(to: vttURL, atomically: true, encoding: .utf8)

JSON Export

let transcript = try await SpeechTranscriber.transcribe(audioURL: url)

let json = try transcript.jsonString()       // pretty, snake_case
let data = try transcript.jsonData()          // for APIs / files
let compact = try transcript.jsonString(prettyPrinted: false)

Progress Reporting

let transcript = try await SpeechTranscriber.transcribe(audioURL: url) { progress in
    print(progress.statusText)               // "Transcribing: 42%"
    progressBar.progress = progress.fraction // 0.0–1.0
}

Error Handling

do {
    let transcript = try await SpeechTranscriber.transcribe(audioURL: url)
} catch TranscriptionError.unavailable {
    print("On-device transcription requires macOS 26 / iOS 26")
} catch TranscriptionError.noAudioTrack {
    print("That video has no audio")
} catch TranscriptionError.localeNotSupported(let id) {
    print("No speech model available for \(id)")
} catch {
    print("Error: \(error.localizedDescription)")
}

How It Works

SpeechTranscriber.transcribe(audioURL:/videoURL:)
  ├─ Fetch       → download remote audio, or extract a video's audio track to .m4a
  ├─ Reserve     → reserve the locale and install its speech model on first use
  ├─ Analyze     → SpeechAnalyzer streams recognized text via AsyncSequence
  ├─ Segment     → NLTokenizer splits text; audioTimeRange gives per-segment timing
  └─ Transcript  → segments + plainText + SRT/VTT/JSON

All transcription happens on-device. The first run for a locale downloads Apple's speech model; every run after that is fully offline. Temporary files are written to the system temporary directory, so the library works inside the App Sandbox without any security-scoped bookmarks.

Models

Property	Type	Description
segments	[TranscriptSegment]	Timed segments in chronological order
audioDuration	TimeInterval	Total audio duration in seconds
transcriptionTime	TimeInterval	Wall-clock time spent transcribing
plainText	String	All segment text joined with spaces
timestampedText	String	"[0:00] Hello everyone", one per line
wordCount	Int	Total words across all segments
speedFactor	Double	Transcription speed vs real time (e.g. 30×)
srt / vtt	String	SubRip / WebVTT subtitle documents

Use Cases

• Podcasts — turn an episode URL into a searchable, quotable transcript
• Interviews — private, on-device transcription for sensitive audio
• Subtitles — batch SRT/VTT for a back-catalogue of videos
• Meeting recordings — recorded calls into editable notes
• Content repurposing — pull quotes and chapters out of long-form audio

Testing

swift test

Unit tests cover time formatting, segment and transcript models, SRT/VTT/JSON export, options, progress, error handling, and input validation. The integration test requires an example.m4a in the test resources directory and an available on-device engine — it skips gracefully when either is missing.

License

MIT License — see LICENSE file for details.

Author

Created by David Sherlock (ArrayPress) in 2026.