A graphical control panel for llama.cpp:
build it, keep it current with upstream, discover models that fit your hardware,
tune every server parameter per model, and run — all from your browser instead
of hand-editing models.ini and long llama-server command lines.
Who it's for: people who want llama.cpp's speed and control but would rather not memorize flags, edit config files by hand, or babysit build commands. It assumes you're comfortable running a setup script once and building llama.cpp for your machine — both guided from the dashboard, on Windows with an NVIDIA GPU (CPU-only works too). Looking for something else? If you want a zero-config, double-click installer with no compile step, LM Studio, Ollama, or Jan will get you running faster — LlamaForge trades that for direct, per-model control over the real llama.cpp server.
LlamaForge is an independent wrapper and is not affiliated with llama.cpp / ggml-org. All inference, model support, and performance come from llama.cpp (MIT, (c) The ggml authors). See NOTICE. Please support the upstream project.
| Tab | What it does |
|---|---|
| Models | Every model on your machine in one list with live GPU VRAM/util/temp meters. Expand a model to edit all ~220 llama.cpp knobs (context, KV-cache type, speculative decoding, tensor split, sampling, rope, ...), grouped and searchable. Save hot-reloads with no restart; load/unload in a click. |
| Discover | Search huggingface.co for GGUF models (newest / most downloaded / most liked). Every quant is rated against your total VRAM - FITS / TIGHT / CPU OFFLOAD - before you download. One click streams the download (multi-shard + vision mmproj handled) and registers it in your registry. |
| Build / Update | Shows your current llama.cpp commit, checks GitHub for how far behind you are, and rebuilds via CMake with flags auto-detected for your CPU/GPU (CUDA arch, AVX-512, quantized-KV flash attention). Prior binaries are backed up; the build streams live. |
| Setup | Checks prerequisites (Git, CMake, Ninja, Python, MSVC, CUDA), installs missing ones with your permission (winget/choco) or links official downloads. Detects hardware and scans all drives for existing GGUF models. Check for deleted models prunes registry entries whose file has since been removed from disk. |
| Setup & hardware detection | Discover with VRAM-fit ratings |
|---|---|
 |
 |
git clone https://github.com/dadwritestech/LlamaForge
cd LlamaForge
powershell -ExecutionPolicy Bypass -File bootstrap.ps1bootstrap.ps1 ensures Python + Git (asking before installing anything), fetches
llama.cpp if you don't have it, writes config.json, and opens the dashboard. From
there: Setup to install any missing compiler/CUDA and scan your drives, Build
to compile llama.cpp for your hardware, Models to tune and run.
Double-click LlamaForge.vbs. It starts the llama.cpp router and the dashboard
hidden, then opens your browser. For autostart, put a shortcut to it in your Startup
folder (Win+R -> shell:startup).
- Dashboard: http://127.0.0.1:8090
- llama.cpp chat UI + OpenAI-compatible API: http://127.0.0.1:8080
To shut everything down, run stop.ps1. It reads the ports from config.json
and stops the dashboard, the router, and every model instance the router spawned:
powershell -ExecutionPolicy Bypass -File stop.ps1- Windows 10/11
- Python 3.10+ (backend is pure stdlib - nothing to
pip install) - NVIDIA GPU for CUDA acceleration (CPU-only builds also supported)
- Everything else (Git, CMake, Ninja, MSVC Build Tools, CUDA) is detected and can be installed from the Setup tab
All machine-specific paths live in config.json (see config.example.json):
| key | meaning |
|---|---|
llama_src |
your llama.cpp git checkout |
build_dir |
CMake build directory |
server_bin |
path to llama-server.exe |
models_ini |
the router preset file LlamaForge edits |
model_dirs |
directories to scan for GGUFs (empty = all fixed drives) |
router_port / panel_port |
ports for llama.cpp and the dashboard |
router_host |
127.0.0.1 (default, local only) or 0.0.0.0 (reachable on your LAN) |
router_api_key |
key clients send as Authorization: Bearer <key>; strongly recommended (and enforceable) whenever router_host isn't 127.0.0.1 |
By default everything binds to 127.0.0.1 only. The Setup tab has a Network
Access panel to opt into serving the llama.cpp API/chat UI to other devices on
your network (e.g. http://192.168.1.x:8080/) and restarts the router for you,
no manual editing needed. A Require an API key toggle (on by default) blocks
LAN access until you set or generate a key; leaving it unchecked exposes the
router unauthenticated. See SECURITY.md.
LlamaForge contains no llama.cpp source code. The backend
(backend/server.py, pure Python stdlib) proxies llama.cpp's own router API, edits
models.ini, and shells out to git / cmake / nvidia-smi / winget. The knob
list is parsed live from llama-server --help, so it stays correct across llama.cpp
versions automatically. HuggingFace downloads are streamed by the backend, so they
work even when llama.cpp is built without SSL.
When models are registered, LlamaForge reads each GGUF's trained context length
straight from its header and writes sensible ctx-size defaults into models.ini
(a 150k global baseline; 100k for models that can't reach it, capped at the
model's own trained length so nothing is over-extended). Per-model settings you set
by hand always win.
LlamaForge is MIT-licensed (LICENSE). It builds and drives llama.cpp - MIT, (c) The ggml authors - see NOTICE and LICENSE.llama.cpp.txt. The hard part is theirs; please star and support the upstream project.