Using a pre-built wheel currently requires specifying the right version, e.g. llama-cpp-python==0.3.4

[eeegnu]

Ran into this problem, and it might be helpful for others if this were clarified in the help text. When I run pip install llama-cpp-python it tries to use 0.3.5, and if that version isn't present in the extra-index-url, it will just ignore it and still try to build the wheel itself.

e.g. for CUDA it reccomends

pip install llama-cpp-python \
  --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu121

but it only worked for me after clicking into the link and seeing that it only supports 0.3.4 / 0.2.9 / ...

[fat-tire]

I think this is because all the CUDA builds broke per issue #1870 and there's been no attempt to rebuild it since. It seems sometimes weeks and weeks go by without a llama-cpp-python update for some reason, though llama.cpp itself might be released multiple times per day. 🤷

[tebjan]

This is indeed a big problem, what would it take to fix the issue? Maybe someone can help? @abetlen

[tebjan]

Windows builds seem to work, I've posted a small update here: #1870 (comment)