Prebuilt windows binaries (i.e. `server.exe`) have a dependency on `llama.dll` - can we remove that?

[jparismorgan]

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• I am running the latest code. Development is very rapid so there are no tagged versions as of now.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new bug or useful enhancement to share.

Current Behavior & Expected Behavior

I downloaded llama-b1606-bin-win-noavx-x64.zip from the latest release and would like to run it in my application. If I run server.exe from the downloaded directory, which has llama.dll next to it, it runs fine. But when I copy over server.exe somewhere else and run it I get:

C:/repo/foo/apps/desktop/llamacpp/engine/server_win32_noavx.exe: 
error while loading shared libraries: llama.dll: cannot open shared object file: No such file or directory

This is an issue because in my app I want to support all variants of avx and acceleration libraries, and with the current configuration I believe I would need to 1) create a new folder for each server.exe variant 2) copy over both server.exe and llama.dll. This is possible, but not as easy as just dealing with standalone server.exe files. So, before doing the workaround proposed above, I was wondering if we could include llama.dll in server.exe?

For reference, this is what Dependency Walker shows for the server.exe from the prebuilt libraries download - note that llama.dll is there:

And this is what it shows if I build for Windows with CMake and no other flags (i.e. follow the README instructions):

The one caveat here is that perhaps I misunderstood the build and llama.dll can be used across avx and acceleration variants? And it's just server.exe which needs to change? Either way, thank you for any help.

Environment and Context

• Physical (or virtual) hardware you are using, e.g. for Linux: Windows
• Operating System, e.g. for Linux: Windows 11

Steps to Reproduce

Please provide detailed steps for reproducing the issue. We are not sitting in front of your screen, so the more detail the better.

1. Download llama-b1606-bin-win-noavx-x64.zip
2. Copy server.exe to another folder
3. Run server.exe

[FiveTechSoft]

Dear Paris,

simply run cmake on the llama.cpp main dir and sln solution for visual studio is created. Then you can build all EXEs with no need for llama.dll. Next I am converting main.exe into a DLL for using it from Windows apps. Working great with Orca2 7B

[jparismorgan]

Hi @FiveTechSoft, are you suggesting I build llama.cpp locally? Because yes, that works. But I'm interested in whether we could update the CI so that the prebuilt Windows binaries don't have a dependencies on llama.dll (i.e. make them not a dynamic library).

[slaren]

There are two reasons why the pre-built binaries use a dynamic library:

• It allows the library to be used for other purposes, such as with llama-cpp-python
• It reduces the size of the archive significantly

I think that your best bet is to build the server binaries yourself.

[jparismorgan]

Thanks @slaren, those are good reasons and I will just build them myself.