llama : fix NaN default tensor split when devices report zero free memory

[jebbadee]

Fixes #24795 ("invalid vector subscript" loading MTP draft models — regression b9553→b9702).

Root cause

When no tensor_split is provided, llama_model_base::load_tensors sizes the default split points by free device memory. If every device reports free == 0 with total != 0 — which Windows/WDDM CUDA commonly does once VRAM is filled, i.e. exactly when a speculative draft model loads second after the target — split_sum is 0 and the normalization computes 0/0 = NaN for every split point. std::upper_bound over NaNs returns past-the-end, and devices.at(layer_gpu) throws out_of_range ("invalid vector subscript" on the MSVC STL), surfacing as error loading model: invalid vector subscript.

This is why the crash looked MTP/arch-specific in #24795: the draft is just the model that loads second. Reproduced with both gemma4-assistant (OP) and qwen35 (Qwen3.8-27B-MTP-ONLY) drafters.

Fix

Guard the zero sum and fall back to an even split (with a warning) instead of dividing by zero.

Verification

Experiments on Windows 11, RTX 4090 Laptop 16 GB, official b10581/b10603 CUDA 12.4 builds, Qwen3.8-27B UD-Q3_K_XL target + a4lg/Qwen3.8-27B-MTP-ONLY-GGUF Q4_K_M draft via --spec-type draft-mtp --spec-draft-model:

condition	result
--ctx-size 512 (VRAM headroom left when the draft loads)	loads
--ctx-size 8192 / 65536	crashes
--ctx-size 8192/65536 + --fit off	still crashes (matches OP)
--ctx-size 8192/65536 + explicit --tensor-split 1 (skips the free-memory branch)	loads
same source, CPU-only build (no devices → branch never runs)	loads

The explicit---tensor-split result isolates the free-memory branch as the crash site; the free==0 + NaN mechanism is the only failure path in it that ends at devices.at(). Patch build compiles and passes a load test (CPU build; no local CUDA toolchain to rebuild the CUDA backend, but the change is backend-agnostic host code).

Full write-up with logs in #24795 (comment).

🤖 Generated with Claude Code

https://claude.ai/code/session_01PwFYMMdiGG5jC3mdZcafxK

[ggml-gh-bot]

Hi @jebbadee, thanks for your contribution!

Per our contribution guidelines, the automated PR checker found the following issue(s) that need your attention:

• PR Template not respected: Please respect the template when creating a new pull request. Make sure to fill out all required sections.

• AI-generated content: While code is allowed to be generated by AI, please write the PR description and commit messages on your own without the help of AI.

---

Please note that maintainers reserve the right to make final decisions on PRs. If you believe there is a mistake, please comment below.