gemma4 12b unified multimodale structure dockerfile pour build rtx5060 ti 16go

[delter86-cell]

J'ai vraiment galéré pour le dockerfile textgen merci a la communauté et léo l'IA de brave.
Dockerfile :

==========================================

STAGE 1: BUILDER

==========================================

FROM nvidia/cuda:12.8.1-cudnn-devel-ubuntu22.04 AS builder

WORKDIR /builder

ARG TORCH_CUDA_ARCH_LIST="12.0"
ARG BUILD_EXTENSIONS=""
ARG APP_UID=1000
ARG APP_GID=1000

RUN DEBIAN_FRONTEND=noninteractive
&& apt-get update
&& apt-get install --no-install-recommends -y
git vim build-essential python3-dev pip bash curl cmake ninja-build
libcurl4-openssl-dev ca-certificates
&& rm -rf /var/lib/apt/lists/*

WORKDIR /home/app/

RUN git clone https://github.com/oobabooga/textgen.git

WORKDIR /home/app/textgen

RUN GPU_CHOICE=A LAUNCH_AFTER_INSTALL=FALSE INSTALL_EXTENSIONS=TRUE ./start_linux.sh --verbose

--- COMPILATION DE LLAMA.CPP (master — support gemma4uv) ---

RUN rm -rf repositories/llama.cpp &&
git clone --depth 1 https://github.com/ggml-org/llama.cpp repositories/llama.cpp &&
cd repositories/llama.cpp &&
cmake -B build -G Ninja
-DCMAKE_BUILD_TYPE=Release
-DGGML_CUDA=ON
-DGGML_NATIVE=OFF
-DGGML_CUDA_FA_ALL_QUANTS=ON
-DCMAKE_CUDA_ARCHITECTURES="120"
-DCMAKE_EXE_LINKER_FLAGS="-Wl,--allow-shlib-undefined"
&& cmake --build build --target llama-server -j$(nproc) &&
chmod +x build/bin/llama-server

Symlink (pas cp) — RPATH $ORIGIN pointe vers build/bin/

RUN mkdir -p /home/app/textgen/installer_files/env/lib/python3.13/site-packages/llama_cpp_binaries/bin/ &&
ln -sf /home/app/textgen/repositories/llama.cpp/build/bin/llama-server
/home/app/textgen/installer_files/env/lib/python3.13/site-packages/llama_cpp_binaries/bin/llama-server

==========================================

STAGE 2: RUNTIME

==========================================

FROM nvidia/cuda:12.8.1-runtime-ubuntu22.04

ARG APP_UID=1000
ARG APP_GID=1000

WORKDIR /home/app/textgen

RUN apt-get update && apt-get install --no-install-recommends -y
bash python3 libcurl4 libstdc++6 libgomp1 &&
rm -rf /var/lib/apt/lists/*

COPY --from=builder --chown=${APP_UID}:${APP_GID} /home/app/textgen /home/app/textgen

COPY --from=builder --chown=${APP_UID}:${APP_GID} /home/app/textgen/repositories/llama.cpp/build/bin/lib*.so* /usr/local/lib/
RUN ldconfig

EXPOSE 7860 5000 5005

USER 1000:1000

CMD ["bash", "-c", "umask 0002 && export HOME=/home/app/textgen && export LD_LIBRARY_PATH=/home/app/textgen/repositories/llama.cpp/build/bin && /home/app/textgen/installer_files/env/bin/python /home/app/textgen/server.py --listen --api --verbose"]

[delter86-cell]

gemma4 12b unified multimodale structure dockerfile pour build rtx5060 ti 16go V1.1

Revu et corrigé, fonctionnel le .sh écrasait tout mettait le bordel entre le conteneur et le dépôt, le build et fond mais au moins on charge gemma4 12B unified avec le multi modale du premier coup .

Pour ceux qui ont une blackwell et qui peuvent pas attendre la version 5 en préparation de textgen

==========================================

STAGE 1: BUILDER

==========================================

FROM nvidia/cuda:12.8.1-cudnn-devel-ubuntu22.04 AS builder

ARG APP_UID=1000
ARG APP_GID=1000

RUN --mount=type=cache,target=/var/cache/apt
--mount=type=cache,target=/var/lib/apt
DEBIAN_FRONTEND=noninteractive &&
apt-get update &&
apt-get install --no-install-recommends -y
git python3 python3-venv python3-dev build-essential
cmake ninja-build libcurl4-openssl-dev ca-certificates &&
rm -rf /var/lib/apt/lists/*

WORKDIR /home/app

RUN for i in 1 2 3; do
git clone https://github.com/oobabooga/textgen.git && break ||
(rm -rf textgen && sleep 5);
done

WORKDIR /home/app/textgen

RUN python3 -m venv installer_files/env &&
installer_files/env/bin/pip install --upgrade pip &&
installer_files/env/bin/pip install torch torchvision torchaudio
--index-url https://download.pytorch.org/whl/cu128

RUN --mount=type=cache,target=/root/.cache/pip
PIP_CACHE_DIR=/root/.cache/pip
installer_files/env/bin/pip install -r requirements/full/requirements_nowheels.txt

RUN git clone --depth 1 https://github.com/ggml-org/llama.cpp repositories/llama.cpp &&
cd repositories/llama.cpp &&
cmake -B build -G Ninja
-DCMAKE_BUILD_TYPE=Release
-DGGML_CUDA=ON
-DGGML_NATIVE=OFF
-DGGML_CUDA_FA_ALL_QUANTS=ON
-DCMAKE_CUDA_ARCHITECTURES="120"
-DCMAKE_EXE_LINKER_FLAGS="-Wl,--allow-shlib-undefined" &&
cmake --build build --target llama-server -j$(nproc) &&
chmod +x build/bin/llama-server

RUN mkdir -p installer_files/env/lib/python3.13/site-packages/llama_cpp_binaries/bin/ &&
ln -sf /home/app/textgen/repositories/llama.cpp/build/bin/llama-server
installer_files/env/lib/python3.13/site-packages/llama_cpp_binaries/bin/llama-server

RUN rm -rf /home/app/textgen/user_data

==========================================

STAGE 2: RUNTIME

==========================================

FROM nvidia/cuda:12.8.1-runtime-ubuntu22.04

ARG APP_UID=1000
ARG APP_GID=1000

WORKDIR /home/app/textgen

RUN --mount=type=cache,target=/var/cache/apt
--mount=type=cache,target=/var/lib/apt
apt-get update &&
apt-get install --no-install-recommends -y
bash python3 libcurl4 libstdc++6 libgomp1 &&
rm -rf /var/lib/apt/lists/*

COPY --from=builder --chown=${APP_UID}:${APP_GID} /home/app/textgen /home/app/textgen
COPY --from=builder --chown=${APP_UID}:${APP_GID} /home/app/textgen/repositories/llama.cpp/build/bin/lib*.so* /usr/local/lib/
RUN ldconfig

EXPOSE 7860 5000 5005

USER 1000:1000

CMD ["bash", "-c", "umask 0002 && export HOME=/home/app/textgen && export LD_LIBRARY_PATH=/home/app/textgen/repositories/llama.cpp/build/bin && /home/app/textgen/installer_files/env/bin/python /home/app/textgen/server.py --listen --api --verbose"]