--model_path Never Work!

[DoiiarX]

Every time i want to use local path, it said that must be a hugging face repo.

CMD ["python", "whisper_online_server.py","--model_dir","/app/models/faster-distil-whisper-large-v3/model.bin", "--backend", "faster-whisper"]

"--model_dir","/app/models/faster-distil-whisper-large-v3/model.bin"

"-model_cache_dir","/app/models/faster-distil-whisper-large-v3/model.bin"

"-model_cache_dir","/app/models/faster-distil-whisper-large-v3"

[Gldkslfmsd]

hi, can you specify what behaviour do you expect and what is wrong? what do you mean by "it said"?

what do you mean by the 3 lines below?

[DoiiarX]

hi, can you specify what behaviour do you expect and what is wrong? what do you mean by "it said"?

what do you mean by the 3 lines below?

Thank you for your response. I'm using a Dockerfile to build my application, and I am encountering issues when specifying the model directory in the Dockerfile's CMD instruction. Here's a clearer explanation of what I'm trying to achieve and the issues I'm facing:

I need to use a local model for Whisper rather than fetching it from a Hugging Face repository each time. However, when I attempt to specify the local model path, the application seems to insist on a Hugging Face repository path instead. Below are the variations of the CMD instruction I tried, all of which resulted in errors indicating that a Hugging Face repository path is required:

1. Directly specifying the path to the model binary:

   CMD ["python", "whisper_online_server.py", "--model_dir", "/app/models/faster-distil-whisper-large-v3/model.bin", "--backend", "faster-whisper"]
   

2. Attempting to use --model_cache_dir with the binary's path:

   CMD ["python", "whisper_online_server.py", "--model_cache_dir", "/app/models/faster-distil-whisper-large-v3/model.bin", "--backend", "faster-whisper"]
   

3. Using --model_cache_dir with just the directory path:

   CMD ["python", "whisper_online_server.py", "--model_cache_dir", "/app/models/faster-distil-whisper-large-v3", "--backend", "faster-whisper"]
   

Could you please provide guidance on how to correctly configure the Dockerfile to use a locally stored model? I'm looking to avoid repeated downloads from the internet due to bandwidth limitations.

Thank you for your assistance.

[DoiiarX]

How could I rewrite this file?

FROM nvidia/cuda:12.0.0-cudnn8-runtime-ubuntu20.04

# 设置工作目录
WORKDIR /app

# 安装系统依赖和Python依赖
RUN apt-get update && apt-get install -y \
    libsndfile1 \
    python3.8 \
    python3-pip \
    git \
 && 
    pip install --no-cache-dir librosa soundfile av faster-whisper ctranslate2==3.24.0

# 克隆whisper_streaming仓库
RUN git clone https://github.com/ufal/whisper_streaming.git

# 切换到仓库目录
WORKDIR /app/whisper_streaming

# 暴露端口，如果有需要的话
EXPOSE 8000

# 运行whisper_streaming服务
CMD ["python", "whisper_online_server.py", "--backend", "faster-whisper"]

[Gldkslfmsd]

the application seems to insist on a Hugging Face repository path instead.

Can you specify from what evidence did you conclude this? What was the error or log message?

Could you please provide guidance on how to correctly configure the Dockerfile to use a locally stored model? I'm looking to avoid repeated downloads from the internet due to bandwidth limitations.
How could I rewrite this file?

I don't know., I can't help with Docker. Make it working inside docker, or locally, then put it to dockerfile.

Maybe you're confusing model.bin file for a dir where model.bin and other necessary files are located?

[DoiiarX]

the application seems to insist on a Hugging Face repository path instead.

Can you specify from what evidence did you conclude this? What was the error or log message?

Could you please provide guidance on how to correctly configure the Dockerfile to use a locally stored model? I'm looking to avoid repeated downloads from the internet due to bandwidth limitations.
How could I rewrite this file?

I don't know., I can't help with Docker. Make it working inside docker, or locally, then put it to dockerfile.

Maybe you're confusing model.bin file for a dir where model.bin and other necessary files are located?

Thank you for your help; I have solved the issue.

The problem was that it required an absolute path and the use of the --model_dir parameter. The final command is:

CMD ["python3", "whisper_online_server.py", "--backend", "faster-whisper", "--host", "0.0.0.0", "--port", "41007", "--model", "large-v3", "--model_dir", "/app/models/faster-distil-whisper-large-v3"]

Thanks to your suggestion, I found the key to resolving the issue.

[Gldkslfmsd]

yes