Time benchmarking script for device-model-compute

[geekodour]

This is not really an issue, but just wanted to share a script that might be helpful if anyone wants to check how much time it takes cpu/gpu per model etc on their own machine.

import itertools
import time
import logging
from faster_whisper import WhisperModel

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s - %(levelname)s - %(message)s",
    datefmt="%Y-%m-%d %H:%M:%S",
)

running_times = {}

def transcribe(device, mod, compute):
    logging.info("processing model: {} device: {} compute: {}\n".format(mod, device,compute))
    model = WhisperModel(mod, device=device, compute_type=compute)

    segments, info = model.transcribe("steve2.wav", beam_size=5)
    logging.debug("Detected language '%s' with probability %f" % (info.language, info.language_probability))
    for segment in segments:
        logging.debug("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))

if __name__ == '__main__':
    devices = "cuda cpu".split()
    models = "small.en medium large-v1 large-v2".split()
    compute = "int8 float16 int8_float16".split()
    for i in itertools.product(devices, models, compute):
        try:
            start = time.time()
            transcribe(*i)
            end = time.time()
            running_times["{}-{}-{}".format(*i)] = round(end - start, 1)
        except ValueError:
            continue
    for k,v in running_times.items():
        print("{} : {}".format(k,v))

sample output

cuda-small.en-int8 : 5.3
cuda-small.en-float16 : 2.5
cuda-small.en-int8_float16 : 2.9
cpu-small.en-int8 : 14.1

@guillaumekln let me know if this will be helpful in this project, I'll cleanup and make a PR, otherwise this issue should be closed.

[guillaumekln]

Thanks! I don't think we need this kind of script in the repo itself, but I'm moving this into a discussion so people can find it in the future.

Couple comments about your script:

• The model loading time should typically not be included in the measured time, since the loading only happens once.
• The time may vary from one run to another depending on what's happening on the system, so you should measure multiple samples for each configuration and take the minimum or average time.