why faster-whisper is slower than openai-whisper?

[guochen-code]

I'm looking for speed up transcription process.

openai-whisper:

start timer

result = model.transcribe(audio_array,language='english',beam_size=1)
print(result["text"])

end timer

faster_whisper:

start timer

segments, _ = model.transcribe(audio_array,language='english',beam_size=1)
print(list(segments)[0].text)

end timer

both are using CPU and 'base.en' model and transcribe a five-word sentence.

The time to complete the transcription:
openai-whisper: ~0.33s
faster-whisper: ~0.5s

is anything wrong here?

[guillaumekln]

• You should compare with the same number of threads. Put this at the very top of your Python script:

import os
os.environ["OMP_NUM_THREADS"] = "4"

• For openai-whisper, beam_size=1 implies best_of=1, so you should add best_of=1 when using faster-whisper.

[guillaumekln]

Even for short audio faster-whisper should be faster.

Did you share the actual code you are running? Because language='english' does not work in faster-whisper. It should be language='en'.

See also this comparison script #279 (comment).

[guochen-code]

Thanks for the lead. language='english' or 'en' works for me in faster-whisper.

faster-whisper is a little bit faster if using the code template from #279. But the main reason is the time for loading the model.
In my case, if remove load_model function outside the run function, openai-whisper is faster.

import os
os.environ["OMP_NUM_THREADS"] = "6"

import whisper
import faster_whisper

import speech_recognition as sr
import numpy as np
import soundfile as sf
import io
import time

model1 = whisper.load_model("base.en", device="cpu")
model2 = faster_whisper.WhisperModel("base.en", device="cpu", compute_type="int8")

def run_openai_whisper(model,audio_array):

result = model.transcribe(
    audio_array,
    beam_size=1,
    best_of=1,
    word_timestamps=False,
    language='english'
)
print(result["text"])

def run_faster_whisper(model,audio_array):

segments, _ = model.transcribe(
    audio_array,
    beam_size=1,
    best_of=1,
    word_timestamps=False,
    language='english'
)
print(list(segments)[0].text)

recognizer = sr.Recognizer()
microphone = sr.Microphone()
with microphone as source:
recognizer.adjust_for_ambient_noise(source)
recognizer.dynamic_energy_threshold = False
time.sleep(1)
print("Listening..............")
audio_data = recognizer.listen(source, timeout=3, phrase_time_limit=3)
wav_bytes = audio_data.get_wav_data(convert_rate=16000)
wav_stream = io.BytesIO(wav_bytes)
audio_array, sampling_rate = sf.read(wav_stream)
audio_array = audio_array.astype(np.float32)
print('audio_array:',audio_array)

run_faster_whisper(model2,audio_array)
start = time.time()
run_faster_whisper(model2,audio_array)
end = time.time()
print("faster-whisper: %f seconds" % (end - start))

run_openai_whisper(model1,audio_array)
start = time.time()
run_openai_whisper(model1,audio_array)
end = time.time()
print("openai-whisper: %f seconds" % (end - start))

Listening..............
audio_array: [ 0.00964355 0.00753784 0.00680542 ... -0.00241089 -0.00506592
-0.00686646]
What is surface back pressure?
What is surface back pressure?
faster-whisper: 0.399812 seconds
What is surface back pressure?
What is surface back pressure?
openai-whisper: 0.298701 seconds

[guillaumekln]

Are you using an old version of faster-whisper? With language='english' I get the following error:

ValueError: english is not a valid language code

Also try removing compute_type="int8" to compare with the same compute type.

[guochen-code]

pip show faster-whisper --->
Name: faster-whisper
Version: 0.6.0
Summary: Faster Whisper transcription with CTranslate2
Home-page: https://github.com/guillaumekln/faster-whisper
Author: Guillaume Klein
Author-email:
License: MIT

No error for 'english' for me (but 'en' also works) and after removed compute_type="int8", here's the result. I would say still within reasonable fluctuation range.

What is surface back pressure?
faster-whisper: 0.314422 seconds
What is surface back pressure?
What is surface back pressure?
openai-whisper: 0.299052 seconds

[Peterdes]

I get similar results on a Colab V100 instance; and it seems JAX isn't faster than either (when using float16).

I guess the explanation might be as follows: I didn't use the original whiper, but instead I used HuggingFace's implementation, which seems to be faster than the original one. Compare: https://github.com/sanchit-gandhi/whisper-jax#benchmarks