20230308: Word-level timestamps visualized

[glangford]

I have experimented with a few videos with the latest release, and wanted to share a short (20s) example of what word level timestamps look like. Note the flashing text that occurs on occasion; this seems to correspond to .srt lines that have no underscores (an artifact of word level emphasis in the transcript). This effect occurs with both hard and soft subtitles added to a video.

tmp-intro.sub.mp4

[glangford]

..but the timing of words is great, and it nails the spelling of the speaker's name!

[jongwook]

This is definitely not intended, I will look for any bugs in the code that writes underlined text. I should also consider making the underscoring optional 😅

[ryanheise]

The problem is not in the SRT generation, actually the result data contains segments where the text and the words in the segment do not match each other. The SRT generator uses the "words" when a word is spoken, but falls back to "text" when there is a pause or gap between words. This is where you see a flash, because the two pieces of data don't match.

[ryanheise]

The flashing is fixed by #1087 .

[glangford]

Here is the .srt, hopefully with some good clues - note that lines 4, 12 and 16 are odd and are inserted right after a punctuation mark - a comma, a period with "Dr." and the period at the end of the sentence.

twp-intro.srt.txt

[dgoryeo]

@glangford , what options/hyper-parameters did you use for the subs of the demo video?

[glangford]

--verbose False --output_format srt --fp16 False --model large --word_timestamps True