difference between Faster-Whisper and WhisperX in word-timestamp

[farzadnk]

I am very curious
why the whisperx Word-timestamp is so accurate
Is it possible for Foster-Whisper to be that accurate?

whisperx repo :
https://github.com/m-bain/whisperX

you can see my result here

whisperx :
https://github.com/guillaumekln/faster-whisper/assets/132581088/c5b07e20-24cd-49ae-964f-5bd64e947520

faster-whisper :
https://github.com/guillaumekln/faster-whisper/assets/132581088/659fa8f1-6994-4666-a5b8-1682e57c15e3

If you know any solution, please share

[guillaumekln]

From the WhisperX README:

Accurate word-level timestamps using wav2vec2 alignment

So they use another model to improve the timestamps.

On the other hand, faster-whisper only relies on the Whisper model and implements the same logic than openai/whisper.

[farzadnk]

Is it possible to implement wav2vec2 for faster-whisper?

[guillaumekln]

whisperX is using both faster-whisper and wav2vec2 so it is possible. But wav2vec2 will not be integrated directly in faster-whisper.