negative timestamp

[sujeongkim]

I occasionally get an AssertionError about negative timestamp from the result.

Traceback (most recent call last):
File "whisper_batch.py", line 67, in
main(args)
File "whisper_batch.py", line 57, in main
transcribe(model=model, audio_path=audio_full_path, output_dir=output_dir)
File "whisper_batch.py", line 34, in transcribe
write_vtt(result["segments"], file=vtt)
File "/data/diva-1/sujeong/miniconda3/envs/whisper/lib/python3.7/site-packages/whisper/utils.py", line 56, in write_vtt
f"{format_timestamp(segment['start'])} --> {format_timestamp(segment['end'])}\n"
File "/data/diva-1/sujeong/miniconda3/envs/whisper/lib/python3.7/site-packages/whisper/utils.py", line 31, in format_timestamp
assert seconds >= 0, "non-negative timestamp expected"
AssertionError: non-negative timestamp expected

The segment causing this issue looks like this (see "start" value below):
{'id': 21, 'seek': 21540, 'start': -743.0200000000001, 'end': 240.82, 'text': '30p.', 'tokens': [2443, 3446, 79, 13, 51635, 51635, 2798, 586, 718, 311, 1286, 322, 281, 264, 1868, 7170, 2799, 11, 293, 550, 321, 603, 747, 257, 574, 412, 264, 51799, 51799], 'temperature': 0.8, 'avg_logprob': -0.7775426228841146, 'compression_ratio': 1.0930232558139534, 'no_speech_prob': 0.2305573672056198}

[jianfch]

Interesting issue. If you can share the audio, it would be easier to figure out where it's wrong.

[jumon]

I get the same problem though it is very rare. As far as I understand, Whisper may output a non-timestamp token after the <transcribe> token even if it is not in the without_timestamps mode. But, the transcribe function assumes that the first decoded token (sliced_tokens[0]) is always a timestamp token, and as a result, start_timestamp_position could be negative if is not the case.
See around this line:

whisper/whisper/transcribe.py

Line 198 in 9f70a35

last_slice = 0

A possible solution would be to suppress generating non-timestamp tokens as a first generated token.

[jumon]

I created a PR to fix the problem by suppressing non-timestamp tokens as a first generated token.
#532