WhisperSource 🎙️

Open-source voice-to-text custom keyboard for iOS

WhisperSource is a privacy-focused, fully open-source iOS custom keyboard that enables voice input across all apps. Built on top of state-of-the-art speech recognition models like NVIDIA's Parakeet v3, it provides fast, accurate transcription while keeping your data under your control.

🌟 Features

• 🎤 Voice Input Anywhere: Use voice typing in any iOS app
• 📱 100% On-Device: All transcription happens locally using CoreML
• 🔒 Zero Network Required: Your voice never leaves your device
• ⚡ Real-time Transcription: Fast, accurate speech-to-text powered by Whisper
• 🌍 90+ Languages: Supports major languages worldwide
• 🔋 Optimized for iOS: Leverages Apple Neural Engine for efficiency
• 📖 100% Open Source: GNU GPL v3 licensed - forever free and open

🎯 Inspiration

WhisperSource draws inspiration from:

• Wispr Flow - Sleek voice input UX
• Superwhisper - macOS voice typing excellence
• CustomKeyboardKit - iOS keyboard extension patterns
• Keyman - Multi-platform keyboard architecture

🏗️ Architecture

WhisperSource/
├── Sources/WhisperSource/
│   ├── Core/
│   │   └── ModelManager.swift        # Multi-model management
│   ├── Models/
│   │   ├── TranscriptionModel.swift  # Protocol for ASR models
│   │   ├── ParakeetV3Model.swift     # NVIDIA Parakeet implementation
│   │   └── [Future: WhisperModel, ConformerModel...]
│   ├── Audio/
│   │   └── AudioRecorder.swift       # Optimized audio capture
│   ├── UI/
│   │   └── KeyboardViewController.swift  # Keyboard extension UI
│   └── Extensions/
│       └── [Helper extensions]
├── Tests/
└── Package.swift

Model Abstraction

WhisperSource uses a protocol-based architecture allowing easy addition of new models:

public protocol TranscriptionModel {
    var name: String { get }
    var version: String { get }
    var supportedLanguages: [String] { get }
    
    func initialize() async throws
    func transcribe(audioData: Data, language: String) async throws -> String
    func transcribeStream(audioStream: AsyncStream<Data>, language: String) -> AsyncStream<String>
    func cleanup()
}

🚀 Getting Started

Prerequisites

• Xcode 15.0+
• iOS 17.0+ target device (WhisperKit requires iOS 17+)
• Swift 5.9+
• ~100MB storage for Whisper base model

Installation

1. Clone the Repository

git clone https://github.com/ksaitor/WhisperSource.git
cd WhisperSource

2. Open in Xcode

open Package.swift
# or create an iOS App project and add this package as a dependency

3. Create iOS App + Keyboard Extension

WhisperSource is a framework. To use it:

1. Create a new iOS App project in Xcode
2. Add a Keyboard Extension target
3. Add WhisperSource as a Swift Package dependency
4. Import and use in your keyboard extension:

import WhisperSource

class KeyboardController: KeyboardViewController {
    // WhisperSource handles the rest!
}

4. Choose Model Size

WhisperSource automatically downloads and uses Whisper models on first run:

// In your keyboard extension:

// Tiny: ~40MB, fastest (good for older devices)
let whisper = WhisperKitModel(modelSize: .tiny)

// Base: ~75MB, recommended (best balance)
let whisper = WhisperKitModel(modelSize: .base)

// Small: ~240MB, more accurate
let whisper = WhisperKitModel(modelSize: .small)

// Medium: ~770MB, highest accuracy (may be slow on older iPhones)
let whisper = WhisperKitModel(modelSize: .medium)

ModelManager.shared.registerModel(whisper)
try ModelManager.shared.setActiveModel(name: "Whisper")

Recommendation: Start with .base for best balance of speed and accuracy.

🎙️ Supported Models

On-Device (100% Private)

• ✅ Whisper (tiny) - 40MB, fastest, great for older devices
• ✅ Whisper (base) - 75MB, recommended, excellent balance
• ✅ Whisper (small) - 240MB, high accuracy
• ✅ Whisper (medium) - 770MB, best accuracy (iPhone 12+ recommended)

All models:

• Run entirely on-device via CoreML
• Support 90+ languages
• Require zero network connection
• Optimized for Apple Neural Engine

Optional: API-Based (Advanced)

For developers who want to use custom backends:

• 🔧 Parakeet v3 - NVIDIA's state-of-the-art (via API)
• 🔧 Conformer - Custom API integration
• 🔧 Wav2Vec 2.0 - Custom API integration

API models require self-hosting or custom server setup.

🔒 Privacy & Security

WhisperSource is designed with privacy as the #1 priority:

• 100% On-Device: All transcription happens on your iPhone/iPad
• Zero Network Access: Works completely offline by default
• No Data Collection: Zero telemetry, no analytics, no servers
• No Cloud: Your voice recordings never leave your device
• Open Source Audit: Every line of code is public and auditable

This means:

• ✅ Works on airplane mode
• ✅ No subscriptions or API keys needed
• ✅ Your conversations stay private
• ✅ No third-party servers can access your voice data

📱 iOS Keyboard Extension Permissions

WhisperSource requires minimal permissions:

• ✅ Microphone Access: To record your voice (iOS standard permission)
• ✅ Text Insertion: To insert transcribed text (built-in keyboard capability)

What WhisperSource does NOT need:

• ❌ "Allow Full Access" - Not required! (because everything is on-device)
• ❌ Network Access - Works completely offline
• ❌ Contacts, Photos, or other sensitive data

This is a major privacy advantage over commercial voice keyboards that require "Allow Full Access" to send your voice to their servers.

🧪 Development

Running Tests

swift test

Adding a New Model

1. Create a new file in Sources/WhisperSource/Models/
2. Implement the TranscriptionModel protocol
3. Register in ModelManager:

class MyCustomModel: TranscriptionModel {
    let name = "My Model"
    let version = "1.0"
    let supportedLanguages = ["en", "es"]
    
    func initialize() async throws { ... }
    func transcribe(audioData: Data, language: String) async throws -> String { ... }
    // ...
}

// Register
ModelManager.shared.registerModel(MyCustomModel())

🤝 Contributing

We welcome contributions! WhisperSource is a community project.

Areas for Contribution

• Models: Add support for new ASR models
• Languages: Improve multi-language support
• UI/UX: Enhance keyboard design and features
• Performance: Optimize audio processing and transcription
• Documentation: Improve guides and examples
• Testing: Add unit and integration tests

Guidelines

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

All contributions must be licensed under GNU GPL v3 to ensure the project remains open source.

📄 License

WhisperSource is licensed under the GNU General Public License v3.0.

This ensures:

• ✅ Anyone can use, modify, and distribute WhisperSource
• ✅ All derivative works must also be open source (copyleft)
• ❌ Cannot be commercialized under proprietary licenses
• ❌ Cannot be used in closed-source products

Why GPL v3?

• Protects user freedom and privacy
• Prevents proprietary forks that lock users in
• Ensures the project benefits everyone forever

See LICENSE for full text.

🎖️ Credits

• NVIDIA NeMo - Parakeet v3 model
• OpenAI - Whisper model
• iOS Developer Community - Keyboard extension patterns
• All Contributors - Thank you! 🙏

🗺️ Roadmap

• Core architecture and model abstraction
• On-device Whisper models via WhisperKit
• Basic keyboard UI
• Multi-language support (90+ languages)
• 100% offline mode
• Real-time streaming transcription
• Custom vocabulary / domain adaptation
• Model size selection UI
• Language switching in keyboard
• Custom wake word support
• TestFlight beta release
• App Store release

📞 Support

• Issues: GitHub Issues
• Discussions: GitHub Discussions
• Email: [TBD]

⭐ Star History

If you find WhisperSource useful, please star the repo! It helps others discover the project.

---

Built with ❤️ by the open-source community

Voice input should be free, private, and open to all.