Studio treats local GGUF+mmproj as a Transformers vision model and fails looking for config.json

[hackxixi]

Summary

Unsloth Studio can successfully load a local GGUF Qwen3.6 model through llama-server, including mmproj and MTP speculative decoding, but then also routes the same .gguf file through the Transformers/Unsloth vision-model path. That second path treats the .gguf file as if it were a Hugging Face model directory/config and fails with No config file found / not a valid JSON file.

The result is that the backend reports a load failure even though the GGUF llama-server path has loaded correctly.

Environment

• Unsloth: 2026.5.7
• Unsloth Zoo: 2026.5.4
• Studio source checkout: eeb49d54 (Bump install.sh / install.ps1 pin to unsloth>=2026.5.7)
• Installed Studio Python: Python 3.13 environment under ~/.unsloth/studio/unsloth_studio
• GPU: NVIDIA GeForce RTX 5090 D
• llama.cpp: unslothai/llama.cpp@b9267
• Model type: local GGUF with adjacent mmproj-F32.gguf
• Model filename pattern: Qwen3.6-27B-UD-Q4_K_XL-MTP.gguf

What I did

In Unsloth Studio, I selected a local GGUF model with an adjacent mmproj file:

.../Qwen3.6-27B-UD-Q4_K_XL-MTP.gguf
.../mmproj-F32.gguf

The backend detected the GGUF and mmproj, then started llama-server with MTP and mmproj:

Detected local GGUF model: .../Qwen3.6-27B-UD-Q4_K_XL-MTP.gguf
Detected mmproj for vision: .../mmproj-F32.gguf
GGUF metadata: context_length=262144
Spec decoding: draft-mtp (MTP-only)
Starting llama-server: .../llama-server -m .../Qwen3.6-27B-UD-Q4_K_XL-MTP.gguf ... --spec-type draft-mtp --spec-draft-n-max 2 ... --mmproj .../mmproj-F32.gguf

llama-server itself loaded successfully:

loaded multimodal model, '.../mmproj-F32.gguf'
speculative decoding context initialized
server is listening on http://127.0.0.1:60351

Actual behavior

After the successful GGUF/llama-server load, Studio starts a Transformers 5.x vision check/load path for the same .gguf file:

Model '.../Qwen3.6-27B-UD-Q4_K_XL-MTP.gguf' needs transformers 5.x -- checking vision via subprocess
Vision check subprocess failed for '.../Qwen3.6-27B-UD-Q4_K_XL-MTP.gguf': {"error": "It looks like the config file at '.../Qwen3.6-27B-UD-Q4_K_XL-MTP.gguf' is not a valid JSON file."}
Loading vision model: .../Qwen3.6-27B-UD-Q4_K_XL-MTP.gguf
Failed to load model: Unsloth: No config file found - are you sure the `model_name` is correct?

Then /api/inference/load returns 500:

Error loading model: Unsloth: No config file found - are you sure the `model_name` is correct?
If you're using a model on your local device, confirm if the folder location exists.
If you're using a HuggingFace online model, check if it exists.
POST /api/inference/load status_code=500

Expected behavior

For a local .gguf model, Studio should keep the load on the GGUF/llama-server path once a GGUF file has been resolved. If mmproj is detected next to the GGUF, that should enable multimodal llama-server flags, but it should not cause the .gguf file itself to be passed into AutoConfig, Transformers, or FastVisionModel as a Hugging Face model directory.

Possible expected behavior:

• Skip Transformers vision detection/loading for resolved local .gguf files.
• Or make the vision check aware of GGUF paths and return the GGUF/mmproj capability without reading the .gguf as JSON/config.
• Or separate is_vision for GGUF/mmproj from is_vision for Transformers model loading.

Why I think this is a path-classification bug

The local Studio source appears to include these relevant paths:

• studio/backend/utils/models/model_config.py
  • detects mmproj and marks the model as vision-capable
  • calls needs_transformers_5(model_name) for Qwen3.6 path strings
  • spawns _is_vision_model_subprocess(...), which expects a Transformers-compatible model ID/path
• studio/backend/utils/transformers_version.py
  • TRANSFORMERS_550_MODEL_SUBSTRINGS includes qwen3.6, so a local GGUF path containing Qwen3.6 is classified as needing Transformers 5.5.0
• studio/backend/core/inference/orchestrator.py
  • subprocess model load can then fail and surface the No config file found error

So the failure seems to be caused by substring-based Transformers-tier detection running on a local .gguf filename/path, after mmproj detection has marked the model as vision-capable.

Notes

MTP itself appears to work. The llama-server log shows:

common_speculative_impl_draft_mtp: adding speculative implementation 'draft-mtp'
speculative decoding context initialized

The problem is the additional Transformers vision path trying to interpret the .gguf file as a JSON config/model directory.

[Imagineer99]

seems valid, will investigate if #5770 addresses this correctly