How to ignore a specific folder inside the --models-dir directory? #21805
|
I'm trying to run some small sized local models in my PC. I use the --models-dir and --models-preset to guide the llama-server where to load models and the model settings. Following is my --models-dir folder structure: I:\LLMMODELS of which EmbeddingModels folder is just a folder containing some embedding models. It is not like other folders like Qwen3.5-2B-Q4_K_M which really represent a model(organize the main model and the projection file together). My question is is there any way to ignore the EmbeddingModels from the model list when starting llama-server? Currently seems it also recogonize it as a model, as following: srv load_models: Loaded 0 cached model presets Following is my preset configuration file: [*] [All-MiniLM-L6-v2-Q4_K_M] [BGE-m3-Q4_K_M] [Jina-Embeddings-v3-Q4_K_M] [Nomic-Embed-Text-v2-Q8_0] [Qwen2.5-VL-3B-Q4_K_M] [Qwen2.5-VL-7B-Q4_K_M] [Qwen3.5-4B-Q4_K_M] [Qwen3.5-2B-Q4_K_M] |
Replies: 2 comments 1 reply
|
Hi @fengzhenqiong , The issue is that llama-server's --models-dir scanner treats every subdirectory as a potential model folder — it doesn't distinguish between a folder that organizes multiple models (like your EmbeddingModels/) and a folder that represents a single multimodal model (like Qwen2.5-VL-3B-Q4_K_M/). There's currently no built-in ignore/exclude flag for --models-dir subdirectories. Your cleanest options are: Option 1 — Move EmbeddingModels out of the scanned directory: Put your embedding models in a separate path like I:\EmbeddingModels\ and reference them directly in your preset config with absolute paths (which you're already doing). Then they won't appear as ghost entries in the auto-scan. Option 2 — Add explicit entries to your preset for the folder: Since your models-preset.ini already has individual entries for each embedding model with full paths, the EmbeddingModels folder entry in the list is harmless — it just has no .gguf at the folder root so it won't load. You can safely ignore it in the UI. Option 3 — Don't use --models-dir at all: Drop --models-dir and rely entirely on your --models-preset file, which already lists every model explicitly with absolute paths. This gives you full control and nothing gets auto-discovered. If this helped you, please mark it as the answer — it helps others in the community who run into the same issue find the solution faster! |
|
Hi @pauldev-hub , Thanks for your details explaination and what you suggests solves my issue. To be honest I still think maybe adding something like .llama-ignore would help more as I like organizing all my models under a single folder(and sub folders). The reason I add individual entries for the embedding models is just because they are not under the root folder and can not be recognized if I dont. As you can see I also have some models in the root folder which I did not configure and just let llama-server to load them with it's default behavior. So what I need to config are just those models which are organized in some sub-folders(like EmbeddingModels) or which contains multiparts(like my Qwen3.5 and qwen2.5-VL models). Anyway this is not something critical, I can add all the entries in my preset file like your Option 3 to make everything under control. Just want to share some thoughts here. Thank you! |
Hi @fengzhenqiong ,
The issue is that llama-server's --models-dir scanner treats every subdirectory as a potential model folder — it doesn't distinguish between a folder that organizes multiple models (like your EmbeddingModels/) and a folder that represents a single multimodal model (like Qwen2.5-VL-3B-Q4_K_M/).
There's currently no built-in ignore/exclude flag for --models-dir subdirectories. Your cleanest options are:
Option 1 — Move EmbeddingModels out of the scanned directory:
Put your embedding models in a separate path like I:\EmbeddingModels\ and reference them directly in your preset config with absolute paths (which you're already doing). Then they won't appear as ghost entries …