Getting this Error :RuntimeError: Failed to load shared library '/venv/lib/python3.10/site-packages/llama_cpp/libllama.so': libgomp.so.1: cannot open shared object file: No such file or directory

[Deepansharora27]

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• I am running the latest code. Development is very rapid so there are no tagged versions as of now.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new bug or useful enhancement to share.

Expected Behavior

The Script Should Run Inside a Container

Current Behavior

I Am Trying to Use Llama CPP Python Inside a LlamaIndex Script within a Docker Container. Here is How I am Doing the Initialisation

llm = LlamaCPP(
    model_path="/usr/src/app/model/llama-2-7b-chat.Q4_0.gguf",
    temperature=0.2,
    max_new_tokens=2000,
    verbose=True,
    context_window=4096,
    messages_to_prompt=messagesToPrompt,
    completion_to_prompt=completionToPrompt,
)

The Model File is Present Inside the /usr/src/app/model directory. Now When I try to Run the Script I Get this Following Error while Spinning up the Container

raise RuntimeError(f"Failed to load shared library '{_lib_path}': {e}")
RuntimeError: Failed to load shared library '/venv/lib/python3.10/site-packages/llama_cpp/libllama.so': libgomp.so.1: cannot open shared object file: No such file or directory

The Same Script When I am Running Locally on My Ubuntu System, it is running fine but running the Script inside a Container is giving me the Above Issue.

[xpq-tech]

Same problem

[TheJimmyNowak]

Same problem here. Problems starts when I import LLamaCPP
from llama_index.llms.llama_cpp import LlamaCPP

[vaibhavmalushte]

Facing the same issue on M3 Mac & when building & deploying on AWS Lambda. Below is my docker file.

FROM python:3.12 as builder
RUN apt-get update && apt-get install -y gcc g++ cmake zip
COPY requirements.txt .
#RUN CMAKE_ARGS="-DLLAMA_BLAS=ON -DLLAMA_BLAS_VENDOR=OpenBLAS -DCMAKE_OSX_ARCHITECTURES=x86_64" pip install --upgrade pip && pip install -r requirements.txt

RUN CMAKE_ARGS="-DCMAKE_OSX_ARCHITECTURES=arm64 -DCMAKE_APPLE_SILICON_PROCESSOR=arm64 -DGGML_METAL=on" pip install --upgrade --verbose --force-reinstall --no-cache-dir pip && pip install -r requirements.txt

FROM public.ecr.aws/lambda/python:3.12

RUN pip install huggingface-hub && \
    mkdir model && \
    huggingface-cli download TheBloke/phi-2-GGUF phi-2.Q4_K_M.gguf --local-dir ./model --local-dir-use-symlinks False && unset CXX
RUN ls -la ./model
COPY --from=builder /usr/local/lib/python3.12/site-packages/ /var/lang/lib/python3.12/site-packages/
COPY lambda_function.py ${LAMBDA_TASK_ROOT}
CMD [ "lambda_function.handler" ]