Skip to content

STREAM ASR v1.5

Latest
Latest

Choose a tag to compare

View all tags
@PanzerPeter PanzerPeter released this 24 Oct 08:23
v1.5
5f220c7

From-scratch streaming English ASR trained on LibriSpeech 960 h on a single RTX 5070: a 55.3 M-parameter Zipformer transducer that decodes offline and streaming (320 ms chunks) from one set of weights, with optional STREAM-LM n-best rescoring.

Results

WER on LibriSpeech, rescoring weights tuned on dev only.

Decoding test-clean offline test-clean streaming test-other offline test-other streaming
greedy 3.23 % 4.60 % 7.88 % 11.12 %
beam 10 3.13 % 4.49 % 7.58 % 10.83 %
beam 10 + LM + ILME 2.57 % 3.43 % 6.57 % 9.01 %
v1.0, beam + LM + ILME 3.43 % 4.45 % 9.17 % 12.24 %

RTF 0.037 offline / 0.072 streaming, 17 ms to first partial.

What changed since v1.0

  • The transducer trained 600k batches over the paired 2x speed-perturbed corpus (24.1 passes over 1,933 h) instead of 175k over the clean one. The language model is unchanged, so the whole gain is acoustic.
  • Seven amplitude bounds, ending with a normaliser on every inter-stack trunk (trunk_norm), are what let a run of that length train without collapsing. MODEL_ARCHITECTURE.md documents each one.
  • The four elementwise leaf modules are compiled with torch.compile (-12.6 % step time, -1 GiB peak VRAM). The full 600k-batch run took about 27 h of GPU.
  • BEST-RQ pretrain overhaul: multi-codebook targets, 300 ms spans, mask fill in CMVN space.
  • Every entry point (--checkpoint, decode.lm_checkpoint) loads the released .safetensors files directly.

Assets

File Contents
model.safetensors transducer, 55.27 M params, average of steps 577.8k to 599.4k
lm.safetensors STREAM-LM, 44.69 M params (byte-identical to the v1.0 file)
bpe500.model SentencePiece BPE-500 tokenizer
SHA256SUMS checksums 
git clone --branch v1.5 https://github.com/PanzerPeter/STREAM-ASR.git && cd STREAM-ASR
uv venv .venv --python 3.12 && uv pip install -r requirements.txt
V=release/stream-asr-v1.5
gh release download v1.5 -D $V -p '*.safetensors' && (cd $V && sha256sum -c SHA256SUMS)
.venv/bin/python -m src.slices.Decode.streaming_decode AUDIO.flac --checkpoint $V/model.safetensors --tokenizer $V/bpe500.model

The v1.5 weights need the v1.5 code: the added trunk normalisers make v1.0 and v1.5 checkpoints mutually incompatible. Details, limitations and provenance are in the model card.

Apache-2.0, weights included. See NOTICE for the attribution requirement.

Assets 6
Loading