7 free in-browser Whisper base tools tested on an hour of audio: repeats and gaps at the 30-second seams

[yaakov123]

I make Mastershot, a set of free browser tools, and one of them runs Whisper base for transcription. I wanted to know how it compares with the other free tools that run Whisper in the browser, so I tested seven of them on the same model (multilingual base), the same computer (M2 Pro, Chrome), and the same recordings. I ran each tool three times and report the middle time.

The recording is LibriVox's The Art of War (public domain), scored against the Project Gutenberg text with Whisper's own EnglishTextNormalizer and jiwer.

On an 8-minute clip, the three fastest tools finished within 3 seconds of each other (31–33 s), and most tools scored a word error rate of 3.5–4.3%. Most of the differences came from what each tool does around the model. For example, one tool's TXT download joins lines with no space between them ("anddeath"), and another leaves out about 150 words.

On 57 minutes, the differences show up where the tools cut the audio into pieces:

Tool	Time	Word error rate	Written twice	Left out
Mastershot	2:03	3.8%	nothing	14 words in 2 places
whisperweb.net	3:21	5.9%	118 words in 7 places	91 words in 6 places
EarScribe	3:26	5.4%	128 words in 9 places	37 words in 3 places

whisperweb.net and EarScribe both work through the recording in overlapping 30-second pieces. Where two pieces meet, a sentence is sometimes written twice and sometimes not at all. "For every victory gained you will also suffer a defeat" comes out twice in both. Mastershot splits the recording at pauses in the speech instead, with no overlap, into up to three parts that are transcribed in parallel. That's why it has no repeats and is faster. It still missed two short stretches.

Everything is downloadable if you want to rerun it or test a tool I missed: the 8-minute clip, the answer keys, the scoring script, and every tool's transcripts.

Full write-up with the method and all the numbers: https://mastershot.app/blog/free-transcription-tools-tested/

If you know a free in-browser tool that runs Whisper base and isn't in the test, tell me and I'll add it.

Jacob