Lemonade does not automatically load models located in extra models dir when accessing via API e.g. from another app

[PCAssistSoftware]

Platform

Linux/Ubuntu

Lemonade Version

10.6.0

GPU / APU Model

not relevant to this issue

Component

llama.cpp

Bug Description

Lemonade does not automatically load models located in extra models dir when accessing via API e.g. from another app

It works fine if model is one downloaded directly within Lemonade interface

But for any located in your extra models dir location, despite working fine directly in Lemonade, if accessed from another app via API then only works if you have preloaded the model within Lemonade, if not you get errors as per details below:

It looks like it is getting confused and trying to download the model despite it already being available.

Log shows:

2026-06-03 11:31:01.217 [Info] (Server) Auto-loading model: Qwen3.6-35B-A3B-MTP-GGUF
2026-06-03 11:31:01.217 [Info] (Server) Model not cached, downloading from Hugging Face...
2026-06-03 11:31:01.217 [Info] (Server) This may take several minutes for large models.
2026-06-03 11:31:01.217 [Info] (ModelManager) Fetching repository file list from Hugging Face...
2026-06-03 11:31:01.344 [Error] (Server) Failed to load model: Failed to fetch model info from Hugging Face API (status: 404)
2026-06-03 11:31:01.344 [Error] (Server) Error 500: POST /v1/chat/completions
2026-06-03 11:31:03.407 [Info] (Server) Auto-loading model: Qwen3.6-35B-A3B-MTP-GGUF
2026-06-03 11:31:03.407 [Info] (Server) Model not cached, downloading from Hugging Face...
2026-06-03 11:31:03.407 [Info] (Server) This may take several minutes for large models.
2026-06-03 11:31:03.407 [Info] (ModelManager) Fetching repository file list from Hugging Face...
2026-06-03 11:31:03.525 [Error] (Server) Failed to load model: Failed to fetch model info from Hugging Face API (status: 404)
2026-06-03 11:31:03.525 [Error] (Server) Error 500: POST /v1/chat/completions
2026-06-03 11:31:07.596 [Info] (Server) Auto-loading model: Qwen3.6-35B-A3B-MTP-GGUF
2026-06-03 11:31:07.596 [Info] (Server) Model not cached, downloading from Hugging Face...
2026-06-03 11:31:07.596 [Info] (Server) This may take several minutes for large models.
2026-06-03 11:31:07.596 [Info] (ModelManager) Fetching repository file list from Hugging Face...
2026-06-03 11:31:07.711 [Error] (Server) Failed to load model: Failed to fetch model info from Hugging Face API (status: 404)
2026-06-03 11:31:07.711 [Error] (Server) Error 500: POST /v1/chat/completions

But in my model list if I toggle showing downloaded only I can see it in list and can manually load it without an issue:

Steps to Reproduce

1. I have several models in my extra_models_dir
2. try to have chat with one of these when connected to Lemonade API from another app e.g. Msty Studio
3. model does not load

Expected vs Actual Behavior

Expected: model to load from extra_models_dir in same way as any other model would load
Actual: model does not load

Log Output

It looks like it is getting confused and trying to download the model despite it already being available.

Log shows:

2026-06-03 11:31:01.217 [Info] (Server) Auto-loading model: Qwen3.6-35B-A3B-MTP-GGUF
2026-06-03 11:31:01.217 [Info] (Server) Model not cached, downloading from Hugging Face...
2026-06-03 11:31:01.217 [Info] (Server) This may take several minutes for large models.
2026-06-03 11:31:01.217 [Info] (ModelManager) Fetching repository file list from Hugging Face...
2026-06-03 11:31:01.344 [Error] (Server) Failed to load model: Failed to fetch model info from Hugging Face API (status: 404)
2026-06-03 11:31:01.344 [Error] (Server) Error 500: POST /v1/chat/completions
2026-06-03 11:31:03.407 [Info] (Server) Auto-loading model: Qwen3.6-35B-A3B-MTP-GGUF
2026-06-03 11:31:03.407 [Info] (Server) Model not cached, downloading from Hugging Face...
2026-06-03 11:31:03.407 [Info] (Server) This may take several minutes for large models.
2026-06-03 11:31:03.407 [Info] (ModelManager) Fetching repository file list from Hugging Face...
2026-06-03 11:31:03.525 [Error] (Server) Failed to load model: Failed to fetch model info from Hugging Face API (status: 404)
2026-06-03 11:31:03.525 [Error] (Server) Error 500: POST /v1/chat/completions
2026-06-03 11:31:07.596 [Info] (Server) Auto-loading model: Qwen3.6-35B-A3B-MTP-GGUF
2026-06-03 11:31:07.596 [Info] (Server) Model not cached, downloading from Hugging Face...
2026-06-03 11:31:07.596 [Info] (Server) This may take several minutes for large models.
2026-06-03 11:31:07.596 [Info] (ModelManager) Fetching repository file list from Hugging Face...
2026-06-03 11:31:07.711 [Error] (Server) Failed to load model: Failed to fetch model info from Hugging Face API (status: 404)
2026-06-03 11:31:07.711 [Error] (Server) Error 500: POST /v1/chat/completions

Additional Context

looking at /models shows me the following:

looks at /models shows me:

{"data":[{"checkpoint":"unsloth/gemma-4-E4B-it-GGUF:Q4_K_M","checkpoints":{"main":"unsloth/gemma-4-E4B-it-GGUF:Q4_K_M","mmproj":"unsloth/gemma-4-E4B-it-GGUF:mmproj-F16.gguf"},"components":[],"created":1234567890,"downloaded":true,"id":"Gemma-4-E4B-it-GGUF","labels":["tool-calling","vision","llamacpp"],"max_context_window":131072,"object":"model","owned_by":"lemonade","recipe":"llamacpp","recipe_options":{},"size":5.56,"suggested":true},{"checkpoint":"/var/lib/models/lmstudio-community/Qwen3.6-35B-A3B-GGUF","checkpoints":{"main":"/var/lib/models/lmstudio-community/Qwen3.6-35B-A3B-GGUF"},"components":[],"created":1234567890,"downloaded":true,"id":"Qwen3.6-35B-A3B-GGUF","labels":["custom","tool-calling"],"max_context_window":262144,"object":"model","owned_by":"lemonade","recipe":"llamacpp","recipe_options":{},"size":19.713079303503036,"suggested":true},{"checkpoint":"/var/lib/models/unsloth/Qwen3.6-35B-A3B-MTP-GGUF","checkpoints":{"main":"/var/lib/models/unsloth/Qwen3.6-35B-A3B-MTP-GGUF","mmproj":"mmproj-F32.gguf"},"components":[],"created":1234567890,"downloaded":true,"id":"Qwen3.6-35B-A3B-MTP-GGUF","labels":["custom","vision","tool-calling"],"max_context_window":262144,"object":"model","owned_by":"lemonade","recipe":"llamacpp","recipe_options":{"ctx_size":262144,"merge_args":true},"size":36.86864176392555,"suggested":true},{"checkpoint":"nomic-ai/nomic-embed-text-v2-moe-GGUF:Q8_0","checkpoints":{"main":"nomic-ai/nomic-embed-text-v2-moe-GGUF:Q8_0"},"components":[],"created":1234567890,"downloaded":true,"id":"nomic-embed-text-v2-moe-GGUF","labels":["embeddings"],"max_context_window":512,"object":"model","owned_by":"lemonade","recipe":"llamacpp","recipe_options":{},"size":0.477,"suggested":true}],"object":"list"}

[korgano]

This is a duplicate of #2014, which should be resolved in #2048.

[PCAssistSoftware]

This is a duplicate of #2014, which should be resolved in #2048.

Excellent, look forward to that version being released

[ramkrishna2910]

Closing as complete.