[Bug] Unsloth Studio built llama.cpp with Vulkan instead of CPU only

[grassoste]

I installed Unsloth Studio with the install script (curl -fsSL https://unsloth.ai/install.sh | sh) on version 2026.7.1 (or 2026.7.2 I did't pay attention) and it worked really well at the first shot! :)

I am running on full CPU, however I have an Intel UHD 630 integrated GPU present (no dedicated GPU).

Yesterday I then upgraded (with the same command) to this versions
Unsloth Version: v0.1.49-beta
Package Version: 2026.7.3
llama.cpp Version: b10063

And Chat prediction started breaking and CPU was not engaging. After some testing and debugging I noticed that llama.cpp (installed by Unsloth) was crashig with the following error: vk::Queue::submit: ErrorDeviceLost / vk::Device::waitForFences: ErrorDeviceLost. I hence realized that llama.cpp was trying to run on the GPU using the vulkan drivers.

I've thus tested the Unsloth version of llama.cpp manually (i.e. stopping unsloth and just running llama-server with the same params of Unsloth logs) changing those params:

• set -ngl 0 (instead of 1) and removing --device Vulkan0: still crashes with the same ErrorDeviceLost. GPU buffer allocations (BO: addr=... vram_only=..., Intel i915 driver) still appear in the log, confirming the Vulkan/GPU backend is still active
• passing GGML_DISABLE_VULKAN=1 to the llama-server command: same exact error.

At this point I tried to swap llama.cpp, so I downloaded the latest version from the official repo, and of course I downloaded the CPU build binaries. Yes the content is a bit different, but Unsloth Studio does not complain too much (just a warning) and calls the llama-server CPU built I just swapped in and it works again as expected (I see high CPU usage and it does not crash even after a few rounds of thinking+tool usage).

Thus it looks like Unsloth Studio correctly detected my integrated GPU and decided to opt for the Vulkan build of llama.cpp. How can I force the CPU build? Or is this a bug and it is not possible? In the docs it says CPU only is possible (and I agree), but I could not find a way to force it during installation or post-installation.
For the moment my workaround seem to work but of course I won't be able to update llama.cpp from Unsloth for now (not such a big problem) and I think the Unsloth version is slightly more optimized.

[VectorCipher]

Thanks for the detailed writeup and for tracking this all the way down to the exact Vulkan error — that made this easy to confirm against the source.

Root cause, confirmed by reading the installer: this isn't a misdetection. install_llama_prebuilt.py deliberately selects the Vulkan prebuilt for any non-NVIDIA/non-AMD GPU it finds — the comment in the code literally says "Intel (or other non-NVIDIA/non-AMD) GPU: use the Vulkan prebuilt." That's normally a reasonable default (Vulkan usually beats CPU on an iGPU), but it crashes outright on your UHD 630 with ErrorDeviceLost, which looks like a driver/hardware-compat issue with that specific iGPU rather than an Unsloth bug per se.

The good news: the override you're asking for already existed internally — there's a fully working --cpu-fallback flag on install_llama_prebuilt.py (force_cpu under the hood) that does exactly what you did manually, just properly. It just wasn't exposed to users; the only prior caller was an automatic arm64-Linux fallback path in setup.sh for a completely different scenario.

Fix, up as a PR: adds a UNSLOTH_LLAMA_CPP_BACKEND=cpu env var, following the same override convention setup.sh already uses for things like UNSLOTH_LLAMA_RELEASE_TAG and UNSLOTH_LLAMA_FORCE_COMPILE. Setting it forwards --cpu-fallback into the installer's prebuilt-selection command, so you'll get the properly matched Unsloth-optimized CPU build instead of needing to hand-swap in the upstream binary as a workaround:

UNSLOTH_LLAMA_CPP_BACKEND=cpu curl -fsSL https://unsloth.ai/install.sh | sh

This should also work as a re-run on an existing install (not just a fresh one), so people who hit this after an upgrade — like you did — don't need a full reinstall.

Two things flagged as separate follow-ups rather than bundled into this fix:

1. The Vulkan ErrorDeviceLost crash itself is worth its own investigation/health-check (e.g. auto-retry with CPU if the GPU backend crashes immediately on launch) so people don't have to notice and self-diagnose this the way you did.
2. A Studio Settings UI toggle for this, once the env var path is proven out.
   In the meantime, your workaround (swapping in the upstream CPU binary) is a reasonable stopgap — just know you'll want to move back to the Unsloth-built one once this lands, since it's more optimized, as you noted.

[danielhanchen]

This should be supported now - sorry on the delay - please update Unsloth Studio to the latest via curl -fsSL https://unsloth.ai/install.sh | sh for Linux, Mac, WSL or irm https://unsloth.ai/install.ps1 | iex for Windows - if it's not supported yet, it'll be in the next Studio release - if it's still not working after that, please open another issue - appreciate it again!

You can now force the CPU llama.cpp build with UNSLOTH_LLAMA_CPP_BACKEND=cpu (added in #7228).

[grassoste]

@danielhanchen Thanks a lot! I think I will wait directly for the next official release of unsloth studio as well to update everything in one go.
I will close the ticket as you implemented a solution to the bug, and as you said I will reopen if it does not work.

[grassoste]

Hi @danielhanchen, I just tested the new command but it does not look like it changes anything. I am now on version unsloth 2026.7.5

I tired both to update from my current version and to delete the .unsloth folder and reinstall from scratch. I tried both the command UNSLOTH_LLAMA_CPP_BACKEND=cpu curl -fsSL https://unsloth.ai/install.sh | sh and UNSLOTH_LLAMA_CPP_BACKEND=cpu curl -fsSL https://unsloth.ai/install.sh | sh -s -- --no-torch, but in all cases, if look at the file UNSLOTH_PREBUILT_INFO.json in .unsloth/llama.cpp is quite clear what Unsloth is pulling:

  "requested_tag": "latest",
  "tag": "b10155",
  "release_tag": "b10155",
  "published_repo": "ggml-org/llama.cpp",
  "asset": "llama-b10155-bin-ubuntu-vulkan-x64.tar.gz",
  "force_cpu": false,

Normally, the manual fix I implemented was to download the CPU version, e.g. llama-b10064-bin-ubuntu-x64.tar.gz.

I can also clearly see the difference when Unsloth loads the model.
With my manual swap of the llama.cpp folder:
"Starting llama-server: /home/stefano/.unsloth/llama.cpp/llama-server -m /home/stefano/.cache/huggingface/hub/models--unsloth--gemma-4-E4B-it-GGUF/blobs/3cf61de12daa015ee0f7b68e7b7c541405bf220e1e942bad8b47cab827d7df80 --port 41345 -c 131072 --parallel 1 --flash-attn on --no-context-shift --alias bfc15c382204943c3a8fff0c750b94ae2364d7a3 --fit on --metrics --jinja --model-draft /home/stefano/.cache/huggingface/hub/models--unsloth--gemma-4-E4B-it-GGUF/blobs/b6a723115efa510d3b3215db1e26790dae84cd08c2134a764f3d194f1f0c3376 --spec-type ngram-mod,draft-mtp --spec-draft-n-max 3 --spec-ngram-mod-n-match 24 --spec-ngram-mod-n-min 48 --spec-ngram-mod-n-max 64 --chat-template-kwargs {\"enable_thinking\": true, \"preserve_thinking\": false}"}

After the update this is the command Unsloth invokes to start llama.cpp (forcing --device Vulkan0):
"Starting llama-server: /home/stefano/.unsloth/llama.cpp/llama-server -m /home/stefano/.cache/huggingface/hub/models--unsloth--gemma-4-E4B-it-GGUF/snapshots/bfc15c382204943c3a8fff0c750b94ae2364d7a3/gemma-4-E4B-it-UD-Q4_K_XL.gguf --port 37475 --parallel 1 --flash-attn on --no-context-shift -c 131072 --alias unsloth/gemma-4-E4B-it-GGUF -ngl -1 --fit off --metrics --slot-save-path /home/stefano/.unsloth/studio/cache/llama-slots --jinja --model-draft /home/stefano/.cache/huggingface/hub/models--unsloth--gemma-4-E4B-it-GGUF/snapshots/bfc15c382204943c3a8fff0c750b94ae2364d7a3/mtp-gemma-4-E4B-it.gguf --spec-type draft-mtp --spec-draft-n-max 2 --chat-template-file /tmp/unsloth_chat_template_sbxo0c7h.jinja --chat-template-kwargs {\"enable_thinking\": true, \"preserve_thinking\": false} --mmproj /home/stefano/.cache/huggingface/hub/models--unsloth--gemma-4-E4B-it-GGUF/snapshots/bfc15c382204943c3a8fff0c750b94ae2364d7a3/mmproj-F16.gguf --device Vulkan0"}

Did I miss something about how to force the cpu backend?

(Btw I did a small benchmark with CPU vs with Vulkan. With my HW, CPU is on average 2.5x/3x time faster than with the Vulkan backend. That's why I prefer to use the CPU backend).

[LeoBorcherding]

thanks for reopening with the prebuilt json and the launch commands, that pins it exactly.

it turns out the override itself works, the issue is where the variable sits in the command. in UNSLOTH_LLAMA_CPP_BACKEND=cpu curl ... | sh, the variable is set for curl, not for the sh on the other side of the pipe, so setup.sh never sees it and installs the auto-detected build (vulkan, for your intel igpu). your second attempt, ... | sh -s -- --no-torch, has the same shape: the variable is still before curl, so it fell back the same way. that lines up with your json exactly, force_cpu: false and the vulkan asset.

putting the variable after the pipe, on the sh side, fixes it:

curl -fsSL https://unsloth.ai/install.sh | UNSLOTH_LLAMA_CPP_BACKEND=cpu sh

or export it first so both sides have it:

export UNSLOTH_LLAMA_CPP_BACKEND=cpu
curl -fsSL https://unsloth.ai/install.sh | sh

worth deleting .unsloth/llama.cpp first so it re-pulls cleanly. after that the json should read force_cpu: true with a cpu asset, and the --device Vulkan0 line should be gone as well, since studio picks the device from the installed build (your manual cpu swap already launched without it).

i've also put up a small installer change (#7675) to add --cpu / --vulkan flags, so this works straight through the pipe without the footgun (... | sh -s -- --cpu). and thanks for the benchmark, cpu being 2.5-3x faster on your hardware is a good reason the override should be easy to reach.