Startup error when deploying TGI with AMD backend on versions `>3.1.0-rocm`

[andrewrreed]

System Info

At startup on TGI >3.1.0-rocm, the text_generation-launcher fails here:

2025-03-24T17:44:27.829150Z  INFO download: text_generation_launcher: Starting check and download process for teknium/OpenHermes-2.5-Mistral-7B
2025-03-24T17:44:27.843241Z ERROR download: text_generation_launcher: text-generation-server not found in PATH
2025-03-24T17:44:27.843270Z ERROR download: text_generation_launcher: Please install it with `make install-server`
Error: DownloadError

To reproduce:

docker run --rm -it --cap-add=SYS_PTRACE --security-opt seccomp=unconfined     --device=/dev/kfd --device=/dev/dri --group-add video     --ipc=host --shm-size 256g  -v $volume:/data     ghcr.io/huggingface/text-generation-inference:3.1.1-rocm     --model-id teknium/OpenHermes-2.5-Mistral-7B

It works fine on 3.1.0-rocm though. It looks like the PATH and VIRTUAL_ENV paths that are set in the base Dockerfile, aren't being set in the Dockerfile_amd:

ENV VIRTUAL_ENV=/usr/src/.venv/
ENV PATH="$PATH:/usr/src/.venv/bin/"

Information

• Docker
• The CLI directly

Tasks

• An officially supported command
• My own modifications

Reproduction

docker run --rm -it --cap-add=SYS_PTRACE --security-opt seccomp=unconfined     --device=/dev/kfd --device=/dev/dri --group-add video     --ipc=host --shm-size 256g  -v $volume:/data     ghcr.io/huggingface/text-generation-inference:3.1.1-rocm     --model-id teknium/OpenHermes-2.5-Mistral-7B

Expected behavior

The launcher should have text-generation-server available in PATH so that it can handle model weights check/download properlly