🎙️ PolyWhisper v0.2.0 — v9 final: 5-language Indic ASR

🎙️ PolyWhisper v9 final — Hindi, Tamil, Telugu, Bengali, Marathi

One frozen Whisper-Small backbone + five tiny LoRA adapters (~14MB each). Per-language augmentation recipe: SpecAugment + speed perturb for Bengali/Marathi only, clean training for Hindi/Tamil.

📊 FLEURS results (beam=1, normalized WER)

Language	v7 baseline	v9 final	Δ
Hindi (hi)	43.0	46.3	+7.7%
Tamil (ta)	68.2	70.1	+2.8%
Telugu (te)	105.9	100.1	✅ −5.5%
Bengali (bn)	198.8	130.2	✅ −34.5%
Marathi (mr)	170.1	96.7	✅ −43.2%

📦 Adapters (all on eulogik/polywhisper)

• hi_best_clean.pt (46.3), ta_best_clean.pt (70.1) — no augmentation
• te/bn/mr_best_prod.pt — SpecAugment + speed perturb

🖥️ ONNX (CPU-only, new)

20 pre-exported graphs under export/onnx/ — fp32 + INT8 encoder/decoder per language (~4× smaller in INT8, fp32 parity < 1e-3 vs torch).

🚀 Use

pip install -e .
polywhisper transcribe audio.wav --lang hi

from polywhisper import transcribe
print(transcribe('audio.wav', lang='mr').text)

CPU-only via ONNX INT8: polywhisper export --lang hi --variant prod --int8

Full benchmarks, FAQ, and training recipe in the README.

PolyWhisper v1.0.0 — Hinglish Code-Switch LoRA Router

PolyWhisper v1.0.0 — Code-Switch ASR with Per-Token LoRA Routing

Hinglish (Hindi-English) ASR on a frozen Whisper-Base encoder + two rank-8 LoRA language experts + 33K-parameter per-token router. MIT licensed. Created by Eulogik (github.com/eulogik · huggingface.co/eulogik).

Benchmarks (3,129-utterance code-switched test set)

System	WER	FuzzyWER	CER	Hallucinations
PolyWhisper v5 router	58.8%	57.3%	57.9%	13
Vanilla Whisper-Base	66.6%	63.3%	67.5%	279
Static 50/50 mix	72.1%	70.4%	71.1%	655

• +13.3 WER pts routing value (vs static 50/50 mix)
• +7.8 WER pts vs vanilla Whisper-Base
• Router per-token language accuracy 89.1%
• Trained in ~a day on 16GB Apple Silicon (M4/MPS) — no GPU cluster

What's in the bundle

• checkpoints/ — en/hi LoRA experts + router (v5, the corrected-label release)
• results/ — full per-utterance eval JSONs (router, static-50/50, vanilla), analysis summary, ortho test set
• LICENSE — MIT

Model card

https://huggingface.co/eulogik/polywhisper-hinglish-router

Reproduce

Full pipeline: https://github.com/eulogik/PolyWhisper (train_v3.py → train_router.py → eval_router.py)

Honest note: absolute WER is far from large-model SOTA — this release demonstrates parameter-efficient per-token routing for code-switched low-resource speech, validated by ablation + weight introspection.

v0.1.0 — Architecture + Pipeline

PolyWhisper v0.1.0

First release. Architecture implemented, tested, and training pipeline validated.

What's included

• PolyWhisper model — frozen Whisper Base encoder + language-specific decoder adapters
• Language ID head — auto-detects language from audio
• Training notebooks — fully resumable, runs on Colab Free (T4 GPU)
• Evaluation notebook — WER computation on FLEURS test set
• Smoke tests — verified on M4 MPS (Apple Silicon)

Architecture

• Encoder: Whisper Base (74M, frozen)
• Per-language adapter: ~20MB (3-layer cross-attention + LoRA)
• Language ID head: 0.1M params
• Trainable params: ~12M total

Training

• Dataset: Common Voice 17.0 (50K samples per language)
• Fully resumable — crash-safe checkpointing every 10 steps
• Saves to Google Drive automatically

Next steps

• Train English + Hindi adapters on Common Voice
• Add Tamil, Telugu, Bengali, Marathi
• HuggingFace model card
• Quantized export for edge deployment