Can't build custom docker image with version > 0.1.64 #408
Description
Prerequisites
Please answer the following questions for yourself before submitting an issue.
- I am running the latest code. Development is very rapid so there are no tagged versions as of now.
- I carefully followed the README.md.
- I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
- I reviewed the Discussions, and have a new bug or useful enhancement to share.
Expected Behavior
RUN pip3 uninstall -y llama-cpp-python
&& CMAKE_ARGS="-DLLAMA_CUBLAS=on" FORCE_CMAKE=1 pip3 install llama-cpp-python --no-cache-dir
Should build the llama-cpp-python successfully
Current Behavior
Install fails with the following message:
CMake Error in vendor/llama.cpp/CMakeLists.txt:
CUDA_ARCHITECTURES is set to "native", but no GPU was detected
Environment and Context
Docker environment, nvidia/cuda:11.8.0-devel-ubuntu22.04
Notably, reverting back to 0.1.64 with the following command fixes it:
RUN pip3 uninstall -y llama-cpp-python
&& CMAKE_ARGS="-DLLAMA_CUBLAS=on" FORCE_CMAKE=1 pip3 install llama-cpp-python==0.1.64 --no-cache-dir
- Physical (or virtual) hardware you are using, e.g. for Linux:
N/A
- Operating System, e.g. for Linux:
nvidia/cuda:11.8.0-devel-ubuntu22.04
- SDK version, e.g. for Linux:
#11 [builder 5/16] RUN python3 --version
#11 0.791 Python 3.10.9
#11 0.791
#11 DONE 0.8s
#12 [builder 6/16] RUN make --version
#12 0.885 GNU Make 4.3
#12 0.885 Built for x86_64-pc-linux-gnu
#12 0.885 Copyright (C) 1988-2020 Free Software Foundation, Inc.
#12 0.885 License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
#12 0.885 This is free software: you are free to change and redistribute it.
#12 0.885 There is NO WARRANTY, to the extent permitted by law.
#12 0.885
#12 DONE 1.0s
#13 [builder 7/16] RUN g++ --version
#13 0.857 g++ (Ubuntu 11.3.0-1ubuntu1~22.04.1) 11.3.0
#13 0.857 Copyright (C) 2021 Free Software Foundation, Inc.
#13 0.857 This is free software; see the source for copying conditions. There is NO
#13 0.857 warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
#13 0.857
#13 0.857
#13 DONE 0.9s
Steps to Reproduce
Please provide detailed steps for reproducing the issue. We are not sitting in front of your screen, so the more detail the better.
my Dockerfile:
FROM nvidia/cuda:11.8.0-devel-ubuntu22.04 as builder
COPY --from=continuumio/miniconda3:4.12.0 /opt/conda /opt/conda
ENV PATH=/opt/conda/bin:$PATH
# Update the base image
RUN apt-get update && apt-get upgrade -y \
&& apt-get install -y git build-essential \
ocl-icd-opencl-dev opencl-headers clinfo \
wget \
&& mkdir -p /etc/OpenCL/vendors && echo "libnvidia-opencl.so.1" > /etc/OpenCL/vendors/nvidia.icd
# Create a new environment
RUN conda create -y -n textgen python=3.10.9
SHELL ["conda", "run", "-n", "textgen", "/bin/bash", "-c"]
RUN pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
RUN git clone https://github.com/oobabooga/text-generation-webui \
&& cd text-generation-webui && git checkout c40932eb39140e039a8a3574497ca94567188ddb && pip3 install -r requirements.txt
RUN pip3 install ninja
RUN pip3 uninstall -y llama-cpp-python \
&& CMAKE_ARGS="-DLLAMA_CUBLAS=on" FORCE_CMAKE=1 pip3 install llama-cpp-python==0.1.64 --no-cache-dir