Skip to content

Releases: hclivess/whisperer

whisperer 1.8.4

Choose a tag to compare

@hclivess hclivess released this 25 Aug 21:00

Standalone builds for Windows x64, Linux x64 and macOS arm64 are attached below (FFmpeg must be on PATH or next to the app).

A styled subtitle keeps its styling through a resync

1.8.3 could read ASS/SSA and MicroDVD but regenerated the output from the cues, so an ASS came back as plain text in whisperer's own style — positioning, fonts, karaoke and comments gone. The original file is now rewritten in place: the two timestamps on each Dialogue: line change, and nothing else in the file does. What comes out is byte for byte the file that went in, with the timing corrected.

  • The mapping from cue back to line is the part that has to be right. Events are matched in the order the parser produced them — by start time — so a file whose lines are out of chronological order still retimes correctly, and a Comment: line is not a subtitle and keeps its own timing.
  • MicroDVD is rewritten at the rate the file declares, because new frame numbers written at a different rate would move every cue.
  • The pass that joins cues too short to read is skipped for a file being retimed in place: it changes the cue count, and a changed count leaves no honest mapping back to the original's lines. If something else changes the count anyway, the retimed file is not written, the log says why, and the formats ticked on the Subtitles tab carry the new timings.
  • Keep an ASS/MicroDVD file's own styling and layout (Sync tab) is on by default; turn it off to regenerate the file in the ticked formats instead.

Verified end to end on a styled file with \pos, \fad, a custom style block and a translator's comment: the output diffs clean against the original once the timestamps are masked, and the timing moved by the offset the fit found.

Everything in 1.8.3 is included.

whisperer 1.8.3

Choose a tag to compare

@hclivess hclivess released this 25 Aug 20:05

Standalone builds for Windows x64, Linux x64 and macOS arm64 are attached below (FFmpeg must be on PATH or next to the app).

Resync reads ASS/SSA and MicroDVD too

1.8.2 started writing both formats and could not read either back, which left anyone who chose them unable to resync their own output. Resync now takes SRT, WebVTT, ASS/SSA and MicroDVD, recognised from the content rather than the file name, so a mislabelled file still opens.

  • ASS is read through its own Format: line, not the usual field order — that order is not fixed between files, and one that moves Start and End would otherwise be read backwards. Comment: lines are not subtitles, brace blocks are styling, \N / \n / \h become real breaks and spaces, and a truncated line costs that one cue rather than the whole file.
  • MicroDVD counts frames, so it needs a rate before it means anything: the rate the file declares on its first line wins, the video's own probed rate stands in, and 25 is the last resort. A wrong rate is a constant ratio over the whole file — which is exactly what the speed fit corrects, the 23.976 ↔ 25 case resync already existed for.
  • Styling is not carried over. Resync rewrites timing and writes the formats ticked on the Subtitles tab, so an ASS keeps its words and loses its positioning and fonts. Retiming a file in place, byte for byte, is a separate job: the merge and minimum-duration passes change the cue count, and that has to be handled deliberately rather than as a side effect.
  • Auto-discovery beside the video and the file dialog cover all four formats, and a .synced. file from an earlier run is never picked up, whatever its extension.

Everything in 1.8.2 is included.

whisperer 1.8.2

Choose a tag to compare

@hclivess hclivess released this 25 Aug 08:59

Standalone builds for Windows x64, Linux x64 and macOS arm64 are attached below (FFmpeg must be on PATH or next to the app).

One dropdown decides what comes out

Deliver as, at the top of the Subtitles tab:

Choice What you get Cost
Subtitle file (default) the formats ticked below — srt, vtt, ass, sub, txt, json none
Embed in the video movie.subbed.mkv / .mp4 — the original streams plus a subtitle track the player can switch on and off a stream copy: quick, nothing re-encoded
Hardcode into the video movie.hardsub.mkv / .mp4 — the subtitles drawn into the picture, impossible to turn off a full x264 re-encode: roughly as long as the film, and some quality goes

It replaces the Embed into video checkbox; settings and presets carrying that checkbox are read as Embed. The container and quality boxes appear only when a video is being written.

Hardcode is for players and sites that show no subtitle tracks at all — anywhere else, embedding keeps the picture untouched and stays switchable.

  • Hardcoding uses FFmpeg's subtitles filter and x264 at CRF 18, 20 or 24 (the box beside the dropdown). The progress bar follows the encode, Stop kills it inside a second, the output is written under a temporary name so nothing half-encoded is left under the real one, and audio is re-encoded only where the container refuses to carry it.
  • FFmpeg is run from a temporary folder holding the subtitles as subs.srt: the filter argument is parsed by FFmpeg rather than the shell, and a real path's colons, backslashes, commas and brackets each break it in their own way. Every other path is made absolute, or it would be resolved against that folder.
  • An audio file has no picture to burn into. The subtitles are written and the video step is skipped with a message, rather than failing the file.

Two more subtitle formats

  • ass — Advanced SubStation Alpha: a styled script with the line breaks placed explicitly.
  • sub — MicroDVD, for older players. It counts frames, not seconds, so it is written at the source's frame rate, probed from the file and declared on its first line; 25 stands in where nothing could be probed, and a file played at another rate drifts by the ratio between them.

Resync still reads .srt and .vtt only — the new formats are written, not parsed.

Everything in 1.8.1 and 1.8.0 is included.

whisperer 1.8.1

Choose a tag to compare

@hclivess hclivess released this 24 Aug 22:09

Standalone builds for Windows x64, Linux x64 and macOS arm64 are attached below (FFmpeg must be on PATH or next to the app).

Fine-tuned models are in the dropdown now

faster-whisper has always loaded any CTranslate2 Whisper model from the Hugging Face Hub, so a language fine-tune was one typed repo id away — but nothing in the app said so and nothing suggested one, which made it a feature only for people who already knew it existed. Eight of them now sit below the sizes in the Model tab:

Model Language
kiendt/PhoWhisper-large-ct2 Vietnamese (VinAI's PhoWhisper)
distil-whisper/distil-large-v3.5-ct2 English, faster than large-v3
avazir/faster-distil-whisper-large-v3-ru Russian
jimmymeister/whisper-large-v3-turbo-german-ct2 German
kotoba-tech/kotoba-whisper-v2.0-faster Japanese
ivrit-ai/whisper-large-v3-turbo-ct2 Hebrew
techiaith/whisper-large-ft-cy-en-ct2 Welsh / English
SoybeanMilk/faster-whisper-Breeze-ASR-25 Mandarin, incl. Mandarin/English code-switching
  • Picking one brings its language with it. A model fine-tuned for Vietnamese left decoding as English is a fine-tune wasted, and it fails quietly — output still appears, it is just worse than plain Whisper would have been. Hebrew and Welsh joined the language list for the same reason.
  • The list is a file, not a table in the source. models.json is written next to the app on first run and read at startup. Entries may be a bare repo id or an id with a label and a language, a local folder holding model.bin works as well as a Hub id, and a broken or emptied file falls back to the built-in list rather than leaving an empty box. The dropdown stays editable for a one-off id.
  • The catalogue is offered under faster-whisper only — whisper.cpp reads GGML files and can load none of it.
{
  "models": [
    {"id": "kiendt/PhoWhisper-large-ct2", "label": "PhoWhisper large — Vietnamese", "language": "vi"},
    {"id": "/models/my-own-ct2-model", "label": "My own fine-tune", "language": "cs"}
  ]
}

The model must be CTranslate2-converted — a plain PyTorch Whisper checkpoint has to go through ct2-transformers-converter first. Repos that ship no tokenizer.json (PhoWhisper is one) fall back to the standard Whisper tokenizer, which is what they were trained with anyway.

Everything in 1.8.0 is included.

whisperer 1.8.0

Choose a tag to compare

@hclivess hclivess released this 24 Aug 02:25

Found by transcribing twelve hour-long lecture recordings end to end with the ordinary defaults and
reading the results file by file; every change was validated by replaying the captured raw decodes of
every previously processed file through the changed code, plus the test suite, before moving to the next.

  • A repaired capital stays repaired. When a false sentence break was removed, the word after it was
    lowered in the cue's text but not in its word list. The moment anything downstream rebuilt the text
    from the words - splitting does - the capital and the break came back; worse, the text/word mismatch
    marked the segment untouchable for every later word-level repair, silently switching them off for
    exactly the cues that needed them. The case change is now mirrored into the word token, under a guard
    that the token really is the word the text lowered. On one lecture this settled a third of the stray
    capitals on its own.

  • Pauses are measured against the audio, not the decoder's claims. Both sentence repairs decide by
    the pause around a full stop or a capital, taken from the word timestamps - and in a stretch where the
    decode has come apart those timestamps lie, writing one-word sentences with half-second gaps over
    continuous speech. Every invented full stop then survived on the evidence of its own invented timing.
    When the VAD's speech regions are available the pause is now the audible silence in the gap: a claimed
    pause the VAD heard speech through counts as no pause. A lecture section that read
    "harmonious interplay Of. Multiple. Patterns right." comes out as prose again.

  • No pause makes "the." a sentence. A full stop after "the" / "an" survived whenever the speaker
    genuinely paused there for emphasis, which speakers do constantly. The article veto now outranks the
    pause everywhere, not only in the no-timestamps fallback.

  • Hotwords learn names, not habits. The vocabulary the first pass hands to the later ones accepted
    any word capitalised once in mid-sentence and seen twice - which let "The", "And", "You" through on
    ordinary decode stumbles. Fed back as hotwords they taught the later passes a style, and both
    verification passes of an hour-long file came apart into comma-separated Title Case identically; the
    1.7.6 guard kept the wreckage out of the file, but the passes bought no verification. A candidate is
    now vetoed by the transcript's own habits - a word the file mostly writes in lower case is not a name,
    and utterance openers and I-contractions never qualify - the same evidence standard the case unifier
    already applies. On the same file the learned list went from "The, And, That's, You, It's, ..." to
    "Husserl, Heidegger, Aries, Eve, Freud, Nietzsche", and the later passes stayed prose.

  • Every pass is compared on the same ground. Candidates were compared over a padded span (the passes
    cut segments in different places), but the transcript's own cue brought only its unpadded text to that
    comparison. On short cues the handicap outweighed the wording: identical healthy passes could score as
    "worded differently", and a real one-word disagreement drowned in neighbour words every pass renders
    the same. All candidates, the base included, are now compared on the cue's territory - the same
    non-overlapping stretch used for replacement text - and the agreement bar is recalibrated for the
    symmetric measure (one word in three differing is a disagreement, one in five is the same reading).
    First lecture: cues settled by agreement rose from 1594 to 1748 of 1864, unresolved fell from 7 to 5.

  • A pass that came apart no longer floods the review list. Its text was already barred from the
    file; it is now also left out of the "worded differently" comparison, so the review list holds real
    disagreements instead of one Title-Case line per cue.

  • The line length limit is hard again. Balancing a cue's text over its lines widened them past Max
    line length
    whenever no word-boundary split fit the line count - a 45-character line under a
    42-character limit, on any awkward two-line cue. The text now takes another line instead, which is
    what the merge logic already promised; only a single word longer than the limit itself may ever
    exceed it.

  • A loop inside a long segment is caught. The repetition folding of 1.7.12 gated on the cue's
    overall rate, so a one-second loop hiding in a thirty-second segment passed as human speech. The
    looped words are now also held to the impossible-rate bar by their own timestamps; a speaker's
    repetition at speaking speed stays untouched either way.

  • Speed and ETA are per pass. Each pass starts its position over at zero, but the clock did not, so
    every pass after the first reported a fraction of its real speed.

whisperer 1.7.12

Choose a tag to compare

@hclivess hclivess released this 23 Aug 22:09

Standalone builds for Windows x64, Linux x64 and macOS arm64 are attached below (FFmpeg must be on PATH or next to the app).

Hallucinations and wrong words

  • The file is decoded more than once and only what the decodes agree on is kept (Passes, Transcription tab, 3 by default). Whisper invents differently every time it is asked but reproduces real speech unchanged, so every extra decode is evidence about the first. Each pass differs from the ones before it — no context carried over, another beam, then sampling temperature — because identical settings would repeat the same computation and prove nothing.
  • Agreement decides, not confidence. Invented text is often fluent and scores well, but two independent decodes hardly ever invent the same words. Whisper's own numbers (avg_logprob, no_speech_prob, compression ratio, and how much speech is really in the span) only pick a winner where no two passes agree at all — and that cue is reported as unresolved rather than quietly trusted.
  • A cue no other pass heard anything under and the VAD finds no speech in is dropped as hallucinated. Deleting a cue takes votes, never a confidence number on its own, and the VAD may only vote for silence when there is next to nothing under the cue by fraction and by seconds.
  • Speech the first pass skipped entirely, but a later pass and the VAD both found, is recovered.
  • Passes 3+ re-decode only the spans nothing agreed on, snapped out to Whisper's 30-second encoder window, so a run of neighbouring disagreements costs one decode rather than ten. Past a quarter of the file it decodes the whole thing again, because that is cheaper. A pass aimed at a few windows only votes inside them, and may never supply text for a cue it only half heard.
  • Cue timing is always the first pass's, and no words are ever rewritten.
  • A Title-Cased reading can never take over a cue. Capitals On Every Word is what a pass looks like when it imitates a prompt or a sample goes wrong, and it can carry a good confidence score while doing it. It is refused even when it wins the vote; where the first pass is the one that came apart, a cleaner rendering of the same words replaces it.
  • Hotwords (Transcription tab): terms to weight the decoder towards. A word decoded as the commoner word it half sounds like is the one failure verification cannot touch, since every pass hears the same audio the same way and all of them agree. Naming the words is the only thing that prevents it. The names the first pass settled on are added automatically for the later passes.
  • Optional review list (Subtitles tab, off by default): writes <name>.review.txt with every span the passes disagreed on and what each of them said, the spans they agreed on but worded differently — where a meaning-changing mis-hearing hides — and the cues the decoder was least sure of. Nothing is rewritten; it is a list of places worth a human eye.
  • A pass that has come apart is judged over the whole decode, not line by line. A decode that breaks down into a single capitalised, full-stopped word per cue scores clean on every individual line; across the whole decode it is unmistakable. Such a pass still votes on whether a cue is real, but may never hand over any text — and if the first pass is the one that came apart, a pass that did not takes over as the transcript, timing and all, since a decode in that state has no timing worth keeping either.
  • Whisper's per-segment quality numbers are kept instead of discarded, and land in the JSON output.

Punctuation, capitals and repeated text

  • Full stops the speaker never made are removed. Whisper punctuates by language model, not by ear, and ends sentences inside phrases. A full stop with less than the configured silence around it is the model's invention: it goes, and the next word returns to lower case. Names, I, abbreviations, initials, decimals and ellipses are never touched, and without word timestamps almost nothing is repaired.
  • A capital with no full stop in front of it is settled. Whisper decodes in 30-second windows and starts each one as if it were a fresh utterance. A real silence before the capital (after a word a sentence can end on) means the speaker did stop, so the full stop is added; anything else lowers the capital, because a capital with nothing in front of it is an error either way.
  • Titles and names are protected by their neighbours, not by the pause: every capital in a title has another capital beside it, while a stray one has lower case on both sides. Only the second kind is lowered.
  • A name gets the case the rest of the file gives it. Whisper capitalises a name in one window and not in the next; nothing in the audio decides it, so the whole file does. A word the file mostly writes in lower case is never touched, a name needs at least two capitalised sightings to count — unless one of them is a possessive, since somebody's is somebody — words that open an utterance (okay, right, well) are never treated as names, and Title-Cased stretches are ignored when reading the file's habits. In English a lower-case i is restored.
  • A phrase repeated faster than speech is folded to one copy. Whisper sometimes latches onto a few words and repeats them until the window ends — real speech underneath, every pass hearing it the same way, so agreement confirms it. Only the clock catches it: the cue claims more words than a mouth can produce in its span, and folding the repeat brings it back to a human rate. A cue that stays impossible afterwards is left alone and named in the log.
  • A sentence the model wrote twice is removed. After a window boundary Whisper sometimes re-emits the sentence it has just written and then carries on, so the copy lies over the words really spoken there. Only the repeated run goes; the cue keeps its real continuation and starts where its own words start. The test is the clock: saying a phrase twice takes roughly the same time both times, while a re-emission is squeezed into whatever room is left, so nothing is removed unless the copy runs at under 60 % of the original, is at least six words long and is near-verbatim.
  • Repeated words and heavy comma use are never treated as defects — people stutter, and a speaker who pauses often has earned the commas. Only the rendering is judged, never the words.
  • Stray foreign-script characters are removed; sentence starts are capitalised.

The window

  • The Live Transcript shows what was written. It fills from the first decode while the file is still being worked on — before the other passes have had their say, before the repairs, the splitting, the snapping and the merging — so until now it showed something that was not what ended up in the file. When a file is finished, its block in the pane is replaced with the cues actually written and marked (written).
  • The icon carries every size the shell asks for — 16 to 128 pixels as bitmaps, 256 PNG-compressed. It used to be a single 256×256 image, and the taskbar, which draws at 16 to 32 pixels, had nothing it could use and fell back to a generic icon.
  • The Windows taskbar shows the app's own icon: the button takes it from the process's Application User Model ID rather than from the window, and without one the process was grouped under whatever launched it and wore that icon instead. The icon file is also found wherever the build puts it.
  • The view follows the work: the cursor moves to the newest line on every update, so the pane scrolls by itself.

Timing

  • No cue is left on screen too short to read. A cue that cannot reach Min cue duration in the free time around it is merged with its neighbour, and the cue layout is a preference rather than a veto: when the joined text does not fit Max lines the cues merge anyway and the text takes an extra line, lines never going over Max line length. The merged cue spans exactly the two it replaces, so no boundary moves and nothing loses sync.
  • Merging runs in every mode, resync included — it only joins two cues into their union, so an imported file keeps all of its text and all of its timing.
  • Min cue duration is enforced one last time before the file is written, in every mode: what merging cannot fix borrows the missing time from a neighbour that has it to spare, and a donor never drops below the minimum itself.
  • Cues are snapped to speech on/offsets from the VAD, held a moment after speech, never overlapping.

Fixes

  • The capital repair used to give up on the whole file at the first segment whose word list did not spell its text — a music cue, a backend oddity — so on a long recording it often did nothing at all. Such a segment is now simply left alone.
  • Three ways multi-pass verification could lose words, all closed: a boundary word belonging to neither cue, the VAD voting a cue away for a low speech fraction, and a windowed pass handing over text for a cue it had only half heard.
  • A replaced cue could repeat a contraction, because a word was carried through the comparison once per token it splits into and a contraction splits into two.
  • A source file given by a bare name, with no directory part, no longer fails with an unexplained FileNotFoundError: ''; the output folder is resolved from its absolute path.
  • Files under temp/ are no longer tracked by git.