Be able to use .gguf models from /path/to/llama.cpp

[nbiish]

Current Setup ✓

1. .llamabarn directory location: /path/to/.llamabarn (on external drive)

2. ~/.llamabarn symlink: Points to /path/to/.llamabarn (just a pointer in your home directory)

3. Model symlinks: All stored in /path/to/.llamabarn/ and point to models in /path/to/llama.cpp/

[nbiish]

Same for the ~/.ollama/ OR /path/to/.ollama 😁

[erusev]

Thanks for the feature request! Just to clarify -- are you trying to avoid re-downloading models that you already have in your llama.cpp directory, or are you looking to use models that aren't in LlamaBarn's catalog?

[nbiish]

I want to avoid re-downloading stuff but also would like to have separate access if i want to sym link a name to the models for another tool to easily access like ollama but some models only work with llama.cpp like the LFM2*.gguf models.

I might be doing something wrong with my ollama implementation, but it easily works with llama.cpp.

I have my own tool in the making named ggufy that sym links all of my llama.cpp and ollama models that auto wraps the cloud models under ollama's stuff and tries to do all this.

It would be cool to see a better version from the ggml-org team through /.llamabarn lol ^^

[andrecardozo-work]

@erusev It would be great to be able to use any .gguf model. so either having a "+" button somewhere where we can paste the hf repo url or be able to point to a local folder to make the model available.

[prawel]

@andrecardozo-work yes, this is a must have. I have my own curated model files on my disk, and i don't want to redownload or symlink.

Maybe something like "default models folder" will solve it?

[erusev]

I'm moving this to Discussions to gather feedback while we keep the issue tracker focused on our current priorities.

[sammcj]

I keep models in one central directory that several tools download into, in the common {publisher}/{repo}/{file}.gguf layout that LM Studio and others use.

The app only discovers models in the Hugging Face cache layout, so none of them show up, and llama-server's own --models-dir doesn't recurse either, so I've been maintaining a symlink farm and a generated router preset just to make the tree visible to it. That's a workaround I'd like to delete when moving from running llama-server manually, to using the llama macOS app.

Scope, to head off the obvious: not Ollama-style support (#88 declined that for good reason), and not hand-editing models.ini (#61 covers that). Just plain .gguf files on disk.

Suggestion: an optional "additional models directory" in Settings. Any .gguf under it (recursively) shows up in the list and loads like any other model. (Probably as read-only?)

Having had a read through the code, I think it's cheap to maintain:

• refreshDownloadedModels() calls the cache scan from a single place, and everything downstream is keyed off the (Model, ResolvedPaths) pairs it returns rather than the cache layout.
• A second scanner returning the same shape plugs into that seam, reusing shard grouping, mmproj and MTP sidecar selection, the non-LLM header filter and quant tag derivation as-is.

Happy to draft a PR if you're open to the idea?

[erusev]

Thanks for the detailed suggestion. I'd pass on it though -- the app and llama.cpp both default to ~/.cache/huggingface/hub, so that's the layout we want to stay on rather than read a second one. If your models live somewhere else, you can point the app at that directory in Settings ("Model directory") -- it just has to be in the HF layout.

It does mean consolidating on the HF layout, and I know that's the part you wanted to avoid.