Skip to content

Speech model v4 (CC-BY-4.0)

Choose a tag to compare

@Techlm77 Techlm77 released this 29 Oct 07:09

Speech model used by NowCaption 1.2.2 and later. This is not an app release.

nowcaption-fastconformer-en-80ms-cc-by-v4.tar.bz2: NowCaption's own fine-tune of NVIDIA's
stt_en_fastconformer_hybrid_large_streaming_multi
(transducer branch, 80 ms lookahead), as an int8 ONNX export for sherpa-onnx. v3 trained further on the
same 3,300 hours of CC0 and CC-BY English speech, with transcripts by NVIDIA's
Parakeet TDT 0.6B v2 (CC-BY-4.0) where the machine
labels v3 learned from leave out words at a clip's edges, and exported with an encoder that skips work
repeated at every step. Same size, tokens and architecture as v3.

  • Word errors on the eight ESB English test sets, through NowCaption's own caption pipeline: 13.04%
    against 13.31% for v3 in continuous use (as the app runs), 12.89% against 13.18% with a fresh stream
    per clip; 12.33% against 12.43% on 400 other clips per set never used to choose it. Accent test set
    (2,992 clips, 20 regions): 13.47% against 13.78%.
  • Less than half the CPU time: 7.2% of one Ryzen 7 7800X3D core for continuous real-time speech
    (v3: 17.7%); words show as soon as with v3 (median 0.204 s, v3 0.210 s). Known limits are in
    benchmark/README.md.
  • License: CC-BY-4.0, © NVIDIA; fine-tuned, exported
    to ONNX and quantized to int8 by NowCaption. Training data, how it was trained and chosen, and the
    files' SHA-256 are in the archive's README.md (the same text as
    MODEL-CARD-v4.md).
  • SHA-256: caad3a84b9547a4ec4143b668cd1d17acf769ada00fa964d80781c124d1598ef