[HOWTO] Easily compile llama.dll without hassle.

[SolsticeProjekt]

So, for anyone reading this who wants a freshly built llama.dll in windows with or without CUDA support,
stop struggling and do what I did:

1. git clone https://github.com/abetlen/llama-cpp-python.git --recurse-submodules
2. cd llama-cpp-python
3. if you want CUDA: set LLAMA_CUBLAS=1
4. pip install .
5. Find the dll in your python's lib/site-packages/llama_cpp

Probably works just as easily in Linux, but idk.

Worked in Win10, python 3.10.6, CUDA 12.2.

[SolsticeProjekt]

So, for anyone reading this who wants a llama.dll in windows with or without CUDA support,
stop struggling and do what I did:

1. git clone https://github.com/abetlen/llama-cpp-python.git --recurse-submodules
2. cd llama-cpp-python
3. if you want CUDA: set LLAMA_CUBLAS=1
4. pip install .
5. Find the dll in your python's lib/site-packages/llama_cpp

Probably works just as easily in Linux, but idk.

Tested using python 3.10.6, CUDA 12.2.

[Green-Sky]

This is somewhat mind boggling, since we provide prebuilt binaries in the release section.
eg: https://github.com/ggerganov/llama.cpp/releases/tag/master-25d43e0

[SolsticeProjekt]

This is somewhat mind boggling, since we provide prebuilt binaries in the release section.

I didn't think I need to be more detailed in my post above. To me it seemed really obvious that this was for people who want to change the code but struggle with compiling it for whatever reason. I know I'm not the only one having issues compiling the whole thing, so this is definitely helpfull to more than just myself.

[checksummaster]

Prebuild seems not include llama.dll ?

[Green-Sky]

Prebuild seems not include llama.dll ?

you are right, only the avx512 version does. I was pretty convinced i pushed the change to dynamic linking to the ci. will look at the pr history and check if i just did not merge it....

[SolsticeProjekt]

Prebuild seems not include llama.dll ?

you are right, only the avx512 version does. I was pretty convinced i pushed the change to dynamic linking to the ci. will look at the pr history and check if i just did not merge it....

Am I supposed to re-open this, now that apparently the underlying problem has surfaced?

[ccbadd]

Is there a pre built HIP version?

[Green-Sky]

Is there a pre built HIP version?

@SlyEcho does the clblast version classify for this? I have no idea.

[SlyEcho]

No, it's referring to the ROCm version. I have not managed it to build it in Windows myself, always get some kind of linker errors or header errors depending on what I try to use.

But it's supposedly very easy: https://github.com/jllllll/llama-cpp-python-cuBLAS-wheels/blob/main/.github/workflows/build-wheel-rocm.yml