[BUG] llama-server binary not found — cannot load GGUF models. Run setup.sh to build it, or set LLAMA_SERVER_PATH

[JibeomPark]

env - mac

If you see this message on Unsloth Studio WebUI, try this.

#maybe first step will be failed because this logs mean that llama-cpp wasn't installed, actually.

1. pip uninstall llama-cpp-python -y
2. CMAKE_ARGS="-DGGML_METAL=on" pip install "llama-cpp-python[server]" --no-cache-dir
3. export LLAMA_SERVER_PATH=$(which llama-server)

If It's not solved, you can try this(I also do this, and it's good to me)

1. git clone https://github.com/ggerganov/llama.cpp
2. brew install cmake
3. cd llama.cpp
4. cmake -B build
5. cmake --build build --config Release -j
6. cp build/bin/llama-server {your_dir}/unsloth_studio/bin/
7. chmod +x {your_dir}/unsloth_studio/bin/llama-server
8. export LLAMA_SERVER_PATH={your_dir}/unsloth_studio/bin/llama-server

[rolandtannous]

@JibeomPark what is the exacct bug you are getting from the install script because unsloth studio setup on mac should install llama.cpp from source (compile).

[JibeomPark]

I use translator

To be precise, no compilation errors occurred during the installation process. However, the error message appears in the WebUI specifically when I try to select a downloaded model after starting the server.

I initially tried to register the llama-server path, but the file was nowhere to be found. I eventually resolved the issue by manually building the binary and adding it to my PATH.

On a side note, I am currently unable to switch between the Studio, Recipes, and Export tabs in the WebUI. Is this a known limitation for Mac support that is planned for a future update?

[rolandtannous]

@JibeomPark the file installs into a central location under ~/.unsloth/llama.cpp

And yes as of now Mac only supports inference until we have full mlx support which is expected soon!

[mrkstu]

I get the same error and don't have the llama.cpp in my .unsloth after installation:

markstewart@Mac studio % ls -la
total 0
drwxr-xr-x 10 markstewart staff 320 Mar 20 13:43 .
drwxr-xr-x 3 markstewart staff 96 Mar 20 13:42 ..
drwxr-xr-x 10 markstewart staff 320 Mar 20 13:42 .venv
drwxr-xr-x 26 markstewart staff 832 Mar 20 13:42 .venv_t5
drwxr-xr-x 3 markstewart staff 96 Mar 20 13:43 assets
drwxr-xr-x 3 markstewart staff 96 Mar 20 13:44 auth
drwxr-xr-x 5 markstewart staff 160 Mar 20 13:43 cache
drwxr-xr-x 2 markstewart staff 64 Mar 20 13:43 exports
drwxr-xr-x 2 markstewart staff 64 Mar 20 13:43 outputs
drwxr-xr-x 2 markstewart staff 64 Mar 20 13:43 runs
markstewart@Mac studio % pwd
/Users/markstewart/.unsloth/studio
markstewart@Mac studio % cd .venv
markstewart@Mac .venv % ls
bin include lib pyvenv.cfg share testing
markstewart@Mac .venv % cd bin
markstewart@Mac bin % ls
accelerate hypothesis rst2html
accelerate-config isympy rst2html4
accelerate-estimate-memory jiwer rst2html5
accelerate-launch json_repair rst2latex
accelerate-merge-weights jsonschema rst2man
activate kernels rst2odt
activate.csh keyring rst2pseudoxml
activate.fish langid rst2s5
Activate.ps1 litellm rst2xetex
chardetect litellm-proxy rst2xml
cyclopts markdown_py ruff
data-designer markdown-it sqlfluff
datasets-cli markdown2 tabulate
ddgs marko tensorboard
diceware mcp tiny-agents
diff-cover normalizer torchfrtrace
diff-quality numba torchrun
diffusers-cli numpy-config tqdm
distro openai transformers
docutils openenv transformers-cfg-cli
dotenv pip transformers-cli
email_validator pip3 trl
evaluate-cli pip3.13 ttx
f2py py.test typer
faker pybind11-config undill
fastapi pyftmerge unsloth
fastmcp pyftsubset uroman
fonttools pygmentize uvicorn
ftfy pygrun watchfiles
get_gprof pytest websockets
get_objgraph python wheel
hf python3 whisper
httpx python3.13
huggingface-cli randomname

[mmathew23]

@JibeomPark @mrkstu can you try removing the ~/.unsloth and unsloth_studio directories? Then try installing again with

curl -fsSL https://unsloth.ai/install.sh | sh
You should see something like

›› curl -fsSL https://unsloth.ai/install.sh | sh

=========================================
   Unsloth Studio Installer
=========================================

==> Platform: macos
==> All system dependencies found.
==> Creating Python 3.13 virtual environment (unsloth_studio)...
Using CPython 3.13.12
Creating virtual environment at: unsloth_studio
Activate with: source unsloth_studio/bin/activate
==> Installing unsloth (this may take a few minutes)...
Using Python 3.13.12 environment at: unsloth_studio
Resolved 80 packages in 401ms
Prepared 6 packages in 102ms
Installed 80 packages in 665ms
 + accelerate==1.13.0
 + aiohappyeyeballs==2.6.1
 + aiohttp==3.13.3
....
 + wheel==0.46.3
 + xxhash==3.6.0
 + yarl==1.23.0
 + zipp==3.23.0
==> Running unsloth studio setup...
╔══════════════════════════════════════╗
║     Unsloth Studio Setup Script      ║
╚══════════════════════════════════════╝
✅ Frontend already built and up to date -- skipping Node/npm check.
finished finding best python
✅ Using python3 (3.13.12) — compatible (3.11.x – 3.13.x)
[====================] 11/11  finalizing
✅ Python dependencies installed

   Pre-installing transformers 5.x for newer model support...
✅ Transformers 5.x pre-installed to /Users/mathew/.unsloth/studio/.venv_t5/

Building llama-server for GGUF inference...
   Building CPU-only (no CUDA detected)...
✅ llama-server built at /Users/mathew/.unsloth/llama.cpp/build/bin/llama-server
✅ llama-quantize available for GGUF export

╔══════════════════════════════════════╗
║           Setup Complete!            ║
╠══════════════════════════════════════╣
║ Launch with:                         ║
║                                      ║
║ unsloth studio -H 0.0.0.0 -p 8888    ║
╚══════════════════════════════════════╝

=========================================
   Unsloth Studio installed!
=========================================

  To launch, run:

    source unsloth_studio/bin/activate
    unsloth studio -H 0.0.0.0 -p 8888

if you're still having issues, please share the output you see after running.

[mrkstu]

That worked. Be aware I originally used the uv install instructions I assume was left by someone on your team on Hacker News, so be aware those are insufficient as is.

[danielhanchen]

This should be fixed now - sorry on the delay - please update Unsloth Studio to the latest via curl -fsSL https://unsloth.ai/install.sh | sh for Linux, Mac, WSL or irm https://unsloth.ai/install.ps1 | iex for Windows - if it's not fixed yet, it'll be in the next Studio release - if it's still not working after that, please open another issue - appreciate it again!