Skip to content

v0.16.0: Embers, several models at once, MCP server, Voice

Choose a tag to compare

View all tags
@dadwritestech dadwritestech released this 06 Oct 19:34
· 188 commits to master since this release
v0.16.0
a0d7e7f

Embers, several models at once, an MCP server, Voice, and a new look.

New

  • Several models at once. Multi-model mode puts your main model on the fastest GPU that fits it and keeps worker models off that GPU. Fit is checked against footprints measured on your machine, and each verdict says whether it's measured, predicted or a rough estimate. Workers are evicted least-recently-used, only after you agree.
  • Per-model builds. Pin a model to any installed llama.cpp build. ik_llama builds run as their own llama-server on 127.0.0.1:8100 and up. GGUF cards say whether a file runs on any build, mainline only or ik_llama only.
  • Embers. Small local agents that watch a topic, keep a wiki and write you a brief on a schedule. A wiki item only counts if it quotes its source verbatim. Model Scout needs no setup; Forge builds a new ember by interviewing you.
  • MCP server. Claude Code, Codex or any MCP client can check status, load and unload models, check fit, search and download from Hugging Face, and hand a task to a local model.
  • Voice. Text to speech through llama.cpp's own llama-tts (Qwen3-TTS or Pocket TTS), plus an OpenAI-style POST /v1/audio/speech. No VRAM is held between requests.
  • pi from Setup. Install, update and remove the pi coding agent from the panel.
  • Stowage, the new default look. Each GPU is drawn as a cargo hold in 1 GiB cells with every loaded model at its measured size; open a model to see where it would go and what wouldn't fit. Hearth and Classic are one click away. Fonts ship with the panel (no CDN).

Fixed

  • Vision projectors pair again (Discussion #16). An mmproj now pairs when its width matches the model's, the same check llama.cpp makes, so Gemma 4, Qwen 3.5 and other current vision models get theirs. <model>-mmproj-*.gguf names count.
  • Logs stay small (#26). The panel reads only the end of a log, and router, vLLM and model logs rotate past 50 MB (7 copies kept), at start and every minute while running.
  • Stats count every loaded model, including ones running as their own process, with live tokens per second each.
  • Out-of-memory with split-mode = tensor explains that tensor split turns --fit off. A busy router port points you at config.json. Docs search reads whole pages.

Full notes: What's New

Install / update:

irm https://raw.githubusercontent.com/dadwritestech/LlamaForge/master/install.ps1 | iex      # Windows
curl -fsSL https://raw.githubusercontent.com/dadwritestech/LlamaForge/master/install.sh | sh  # Linux / macOS

Early preview: Windows + NVIDIA is the most-tested path; Linux and macOS pass CI but have had little real-hardware use.

Assets 2
Loading