GPU CUDA Docker Image

[ahmetoner]

Hello All,

As we announced before our Whisper ASR webservice API project, now you can use whisper with your GPU via our Docker image.

Whisper ASR Webservice now available on Docker Hub. You can find the latest version of this repository on docker hub for CPU and GPU.

Docker Hub: https://hub.docker.com/r/onerahmet/openai-whisper-asr-webservice

For CPU:

docker run -d -p 9000:9000 -e ASR_MODEL=base onerahmet/openai-whisper-asr-webservice:latest

For GPU:

docker run -d --gpus all -p 9000:9000 -e ASR_MODEL=base onerahmet/openai-whisper-asr-webservice:latest-gpu

# Interactive Swagger API documentation is available at http://localhost:9000/docs

Available ASR_MODELs are tiny, base, small, medium and large

For English-only applications, the .en models tend to perform better, especially for the tiny.en and base.en models. We observed that the difference becomes less significant for the small.en and medium.en models.

# Source: https://github.com/ahmetoner/whisper-asr-webservice/blob/main/Dockerfile.gpu
FROM nvidia/cuda:11.7.0-base-ubuntu20.04

ENV PYTHON_VERSION=3.9
ENV POETRY_VERSION=1.2.0
ENV POETRY_VENV=/app/.venv

RUN export DEBIAN_FRONTEND=noninteractive \
    && apt-get -qq update \
    && apt-get -qq install --no-install-recommends \
    python${PYTHON_VERSION} \
    python${PYTHON_VERSION}-venv \
    python3-pip \
    ffmpeg \
    && rm -rf /var/lib/apt/lists/*

RUN ln -s -f /usr/bin/python${PYTHON_VERSION} /usr/bin/python3 && \
    ln -s -f /usr/bin/python${PYTHON_VERSION} /usr/bin/python && \
    ln -s -f /usr/bin/pip3 /usr/bin/pip

RUN python3 -m venv $POETRY_VENV \
    && $POETRY_VENV/bin/pip install -U pip setuptools \
    && $POETRY_VENV/bin/pip install poetry==${POETRY_VERSION} \
    && $POETRY_VENV/bin/pip install torch==1.13.0+cu117 -f https://download.pytorch.org/whl/torch

ENV PATH="${PATH}:${POETRY_VENV}/bin"

WORKDIR /app

COPY . /app

RUN poetry config virtualenvs.in-project true
RUN poetry install

ENTRYPOINT ["gunicorn", "--bind", "0.0.0.0:9000", "--workers", "1", "--timeout", "0", "whisper_asr.webservice:app", "-k", "uvicorn.workers.UvicornWorker"]

[grimmer0125]

@ahmetoner
I got this WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested. It will be helpful if there is a universal docker image for mac ARM users. I'll try to build the docker image locally, and thank you for supplying these.

[grimmer0125]

I also note that it is not supporting .m4a, is ffmpeg possible to support it?

[ahmetoner]

Hello @grimmer0125,

Arm based docker image version released. You can find it from there under tags page:
https://hub.docker.com/r/onerahmet/openai-whisper-asr-webservice

Repository address: https://github.com/ahmetoner/whisper-asr-webservice

We will create a new version to support m4a. (ahmetoner/whisper-asr-webservice#42)

[grimmer0125]

@ahmetoner Thank you so much. It is awesome. A quick question, does the added ffmpeg code affect any audio quality and the subsequent transcript quality?

[vmatekole]

If you want to enable gpus and you are using docker-compose. This configuration worked for me.
Add it at the root level of your service:

    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [ gpu ]