Repository navigation
Oído v2: faster than speech on one ESP32-S3 core
Version 1 ran at 1.8-2.0x slower than speech on the chip. Version 2 runs at 0.54 (variant B) and 0.72 (variant A) of the audio duration with the on-chip language model, and the final text appears 259 ms (B) / 431 ms (A) after the end of speech (audio fed at real-time speed; 178-634 ms on 15-89 s of continuous speech). Measured on a physical board (ESP32-S3 N16R8, one core, 240 MHz); the board's transcripts equal the host build's on 18 of 18 clips.
| LibriSpeech test-clean / test-other, streaming 0.64 s chunks + language model | WER % |
|---|---|
| A (16 blocks, 7.7 MB int4) | 7.1 / 16.4 |
| B (12 blocks, 5.9 MB int4) | 9.8 / 21.0 |
Streaming costs accuracy: test-other stays above 10 % in every streaming mode (1.28 s chunks: A 6.0 / 14.0). English only. The firmware's end-of-speech wait (0.8 s by default) comes on top of the delays above.
Files
oido_v2a.tnm: variant A, int4 (best accuracy) ·oido_v2b.tnm: variant B, int4 (fastest) ·oido_v2.tlm: language model · all CC-BY-SA-4.0 (see NOTICE)- also on Hugging Face: https://huggingface.co/lokutor-ai/oido-v2-realtime-int4
oido-paper.pdf: the paper (v2), https://arxiv.org/abs/2610.13447
Try it
git clone https://github.com/lokutor-ai/oido && cd oido/esp32/host && make
python live_demo.py --model ../../models/oido_v2a.tnm # laptop microphone, same engine as the chip
../tools/flash.sh /dev/ttyUSB0 ../../models/oido_v2a.tnm # ESP32-S3-DevKitC-1 N16R8 + INMP441 microphone