LLM Suite Building the Engine

Building the Engine

RulZ runs models with llama-server.exe from llama.cpp. It starts one llama-server process per loaded model, each on its own port bound to 127.0.0.1, and talks to it over HTTP. This keeps every model's settings, log and crashes separate.

You can use a prebuilt Windows release from the llama.cpp releases page, or build your own for the best match to your GPU.

Building with CUDA

You need Visual Studio 2022 (with the C++ workload), CMake and the NVIDIA CUDA Toolkit. From a clone of llama.cpp:

cmake -S C:\coding\llama.cpp -B C:\coding\llama.cpp\build-cuda -G "Visual Studio 17 2022" -DGGML_CUDA=ON -DCMAKE_CUDA_ARCHITECTURES="89;120" -DLLAMA_BUILD_SERVER=ON -DLLAMA_CURL=OFF
cmake --build C:\coding\llama.cpp\build-cuda --config Release --target llama-server -j

The CUDA build takes a while; 20 to 40 minutes is normal.

CMAKE_CUDA_ARCHITECTURES should list the GPUs you own:

GPU generation	Value
RTX 20 series	75
RTX 30 series	86
RTX 40 series	89
RTX 50 series	120

Separate multiple values with a semicolon, as in the example above for a machine with a 40- and a 50-series card. Use a separate build folder (such as build-cuda) so an existing CPU build stays untouched.

Check the result:

C:\coding\llama.cpp\build-cuda\bin\Release\llama-server.exe --list-devices

It should list every GPU you expect.

Telling RulZ where it is

RulZ looks, in order, in:

1. the folder set in Settings → Engine (a folder containing llama-server.exe, or the file itself);
2. C:\coding\llama.cpp\build-cuda\bin\Release and ...\build-cuda\bin;
3. C:\coding\llama.cpp\build\bin\Release and ...\build\bin.

Use Settings → Engine → Browse or type the folder, then press Test. The card shows the engine version and the GPUs it found, or the exact error if it could not start.

DLLs and the PATH

A program started from Explorer gets a smaller PATH than one started from a terminal, so DLLs that happen to be found in your terminal may be missing for the app. RulZ helps by adding these folders, when they exist, to the PATH of every llama-server it starts:

• the CUDA runtime: %CUDA_PATH%\bin\x64 and %CUDA_PATH%\bin;
• OpenSSL: C:\OpenSSL-Win64\bin, C:\Program Files\OpenSSL-Win64\bin and C:\Program Files\OpenSSL\bin.

If your DLLs live somewhere else, the simplest fix is to copy them next to llama-server.exe (for OpenSSL, libcrypto-3-x64.dll and libssl-3-x64.dll). If llama-server still fails to start with Windows error 0xC0000135, see Troubleshooting.

GPU numbering

Child processes get CUDA_DEVICE_ORDER=PCI_BUS_ID, so "GPU0" and "GPU1" in RulZ match the numbering in nvidia-smi. Without this, CUDA numbers faster cards first, which can differ from nvidia-smi on machines with two different GPUs.

Keeping the engine up to date

New llama.cpp versions add model architectures and speed-ups, and RulZ passes recent flags such as --spec-type for speculative decoding. If a new model fails to load with an "unknown architecture" error, update llama.cpp, rebuild, and press Test in Settings → Engine.