Problem with importing llama-cpp-python library on Linux CentOS

[Leviathan-l]

It's my first bug report on the GitHub, please, don't be mad if I've done something wrong.

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• [☑️] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
• [☑️] I carefully followed the README.md.
• [☑️] I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• [☑️] I reviewed the Discussions, and have a new bug or useful enhancement to share.

Expected Behavior

I'm new to LLM and wanted to try it locally. So I found hugging_face and langchain, which I used in my implementation then. So I expected that there would be no problem with the import, especially after making sure that the llama-cpp-python library, which is used in mentioned frameworks, was installed successfully.

Current Behavior

But every time when compiler comes to the line with LlamaCpp object I got error ImportError: Could not import llama-cpp-python library. Please install the llama-cpp-python library to use this embedding model: pip install llama-cpp-python. I tried to upgrade it with following command pip install llama-cpp-python --upgrade --force-reinstall --no-cache-dir, but it didn't help.

Environment and Context

• I am using Virtual Machine with following properties:

CentOS Linux release 8.5.2111

Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 1
Core(s) per socket: 8
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 85
Model name: Intel Xeon Processor (Cascadelake)
Stepping: 5
CPU MHz: 2095.076
BogoMIPS: 4190.15
Hypervisor vendor: KVM
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 4096K
L3 cache: 16384K
NUMA node0 CPU(s): 0-7

Python 3.10.9
GNU Make 4.2.1
g++ (GCC) 8.5.0 20210514 (Red Hat 8.5.0-4)
langchain==0.1.1
langchain-community==0.0.12
llama-cpp-python==0.2.28

Failure Information (for bugs)

ImportError: Could not import llama-cpp-python library. Please install the llama-cpp-python library to use this embedding model: pip install llama-cpp-python

Steps to Reproduce

1. Create venv with Python 3.10.9

2. Import LlamaCpp through langchain_community or langchain. Import hf_hub_download from huggingface_hub

3. Declare

    downloaded_model_path = hf_hub_download(repo_id="TheBloke/Mistral-7B-Instruct-v0.1-GGUF",
                                    filename="mistral-7b-instruct-v0.1.Q4_0.gguf")
   
    llm = LlamaCpp(
      model_path=downloaded_model_path, 
       n_ctx=2048,
       n_threads=10,
       n_gpu_layers=25,
       temp=0.1,
       n_batch = 512, 
       n_predict = -1, 
      n_keep = 0
    )
   

4. Get the error.

Note: Many issues seem to be regarding functional or performance issues / differences with llama.cpp. In these cases we need to confirm that you're comparing against the version of llama.cpp that was built with your python package, and which parameters you're passing to the context.

Try the following:

1. git clone https://github.com/abetlen/llama-cpp-python
2. cd llama-cpp-python
3. rm -rf _skbuild/ # delete any old builds
4. python -m pip install .
5. cd ./vendor/llama.cpp
6. Follow llama.cpp's instructions to cmake llama.cpp !Errors on this step! Seems like I need CMake >3.21, and mine is 3.20.2. Encounter errors with ./bootstrap while upgrading. Can't resolve it...
7. Run llama.cpp's ./main with the same arguments you previously passed to llama-cpp-python and see if you can reproduce the issue. If you can, log an issue with llama.cpp

[abetlen]

@Leviathan-l did you have an issue installing llama-cpp-python ie with pip install llama-cpp-python?

[Leviathan-l]

@abetlen no, pip install llama-cpp-python worked as expected and the binding was installed successfully but when I try to execute a piece of code above, I encounter the import error mentioned earlier. Don't really have any thoughts why it happens...
I should also say, that installing langchain via anaconda venv helped me somehow and I haven't seen this error after that. I can provide any additional information about this venv a bit later, when I can have an access to this machine again. But I don't really know what exactly would be helpful - python/pip version? Maybe smth else?