Llama.cpp (and ik_llama) Server Launcher

A user-friendly GUI (Tkinter) to easily configure and launch the llama.cpp and ik_llama server, manage model configurations, set environment variables, and generate launch scripts.

This python script provides a comprehensive graphical interface for llama.cpp and ik_llama's server, simplifying the managing of command-line arguments and models.

✨ Key Features

• Intuitive GUI: Easy-to-use Tkinter interface with tabbed sections for:
  • Main Settings (paths, model selection, basic parameters)
  • Advanced Settings (GPU, memory, cache, performance, generation)
  • Chat Template (select predefined, use model default, custom Jinja, plus reasoning / thinking controls)
  • Environment Variables (manage CUDA and custom variables)
  • MTP / Speculative Decoding (draft model picker, cross-backend draft controls)
  • Build (beta) (build llama.cpp / ik_llama from source with a full CMake-flag editor and saved build profiles)
  • HF Downloader (browse and download models from the Hugging Face Hub)
  • Configurations (save/load/import/export launch setups)
  • Settings (theme, fonts, Windows high-DPI scaling, managed Python venv)

📸 View Advanced Settings Screenshot

• Comprehensive Parameter Control: Fine-tune your llama.cpp server:

  • Model Management: Scan directories for GGUF models, automatic model analysis (layers, architecture, size) with fallbacks, manual model info entry.
  • Vision/mmproj Handling: Automatic mmproj detection, with an mmproj dropdown shown when multiple projector files are found for the selected model.
  • Core Parameters: Threads (main & batch), context size, batch sizes (prompt & ubatch), sampling (temperature, min_p, seed).
  • GPU Offloading: GPU layers (with override), tensor split (with VRAM-based recommendations), user-orderable GPU list, main GPU selection, Flash Attention toggle.
  • MoE Offload: --cpu-moe / --n-cpu-moe controls for offloading expert layers to CPU on both backends.
  • Context Fit: --fit (auto-size context to available VRAM) and --parallel slot configuration.
  • Memory & Cache: KV cache types (K & V, including Q6_K), mmap, mlock, no KV offload.
  • Network: Host IP and port configuration.
  • Generation: Ignore EOS, n_predict (max tokens).
  • Custom Arguments: Pass any additional llama.cpp server parameters.
  • ik_llama Support: Dedicated parameters tab for ik_llama-specific options.

  📸 View ik_llama Screenshot

  

• System & GPU Insights:

  • Detects and displays CUDA GPU(s) (via PyTorch), system RAM, and CPU core information.
  • Supports manual GPU configuration if automatic detection is unavailable.

📸 View Chat Templates Screenshot

• Chat Template Flexibility:
  • Load predefined chat templates from config/chat_templates.json.
  • Option to let llama.cpp decide the template based on model metadata.
  • Provide your own custom Jinja2 template string.
  • --jinja toggle for Jinja template rendering (required by many modern instruction-tuned GGUFs).
  • Reasoning / Thinking controls: --reasoning (on / off / auto), --reasoning-format (none / deepseek / deepseek-legacy / auto), --reasoning-budget, --reasoning-budget-message, and advanced --chat-template-kwargs passthrough.

📸 View MTP / Speculative Decoding Screenshot

• MTP / Speculative Decoding:
  • Enable speculative decoding with cross-backend support (llama.cpp and ik_llama).
  • Speculative type selector (e.g., draft-simple, draft-mtp) with sensible default prefill per type.
  • Common draft controls: n-max, n-min, p-min, p-split, plus a one-click Reset to defaults.
  • Draft model GGUF picker with smart draft GPU controls (-ngld, per-device -devd selection, draft K/V cache types, --spec-draft-cpu-moe / n-cpu-moe).
  • MTP enforces --parallel 1 automatically; optional --no-mmproj for MTP GGUFs that embed an unused vision projector.

📸 View Build Tab Screenshot

• Build from Source (beta): Build llama.cpp or ik_llama directly from the launcher — no separate CMake wrangling required.
  • Backend + source management: Pick the backend (kept in sync with the launcher), auto-clone the upstream repo if the source directory is missing, and check out any branch / tag / commit. Optional git pull --ff-only before building, plus an update banner with Check, Pull only, and Pull & Rebuild actions.
  • Toolchain auto-detection: Enumerates every CUDA toolkit on disk, nvcc, versioned CC/CXX compilers (gcc-9…15, Homebrew gcc, Apple clang, MSYS2/MinGW), plus cmake, ninja, ccache, and git with versions. A one-click helper installs missing CMake / Ninja / Git with an OS-aware command (apt/dnf/pacman/zypper/apk/brew, winget/choco/scoop).
  • CUDA architecture picker: Auto-detect installed GPUs (via nvidia-smi, falling back to PyTorch) to populate CMAKE_CUDA_ARCHITECTURES, or choose by generation (Kepler → Blackwell) from a validated catalog, with -a (arch-accelerated) and -f (family-forward) variant toggles.
  • Full CMake-flag editor: Backend-aware, grouped controls for build targets (server, Web UI, tools, OpenSSL/CURL), CUDA (FlashAttention, graphs, MMQ, NCCL, compression), ik_llama IQK CPU kernels, alternate GPU backends (Vulkan, HIP/ROCm, Metal, SYCL, OpenCL, MUSA, RPC), the full x86 SIMD matrix (AVX2/AVX-VNNI/AVX512…), BLAS vendors, and LTO/ccache/OpenMP. An "Optimized for this system" preset tunes a sensible set automatically.
  • Live preview & in-app build: See the exact resolved cmake -D… invocation, Copy command or Save as .sh, then Start build to stream staged git/configure/build output into a color-coded console (cancel, auto-scroll, save log). Auto-recommended parallel job count and Ninja / Unix Makefiles generator selection.
  • Saved build profiles: Name, save, and reload complete build configurations (backend, dirs, git ref, archs, every flag) to build_configs.json.
  • Managed Python venv: Create and manage a launcher-owned virtual environment (Settings tab) — install/update/remove requests, torch, psutil, and huggingface_hub per venv, with safety guards against deleting anything that isn't actually a venv.

📸 View Hugging Face Downloader Screenshot

• Hugging Face Downloader: Browse and pull models from the Hugging Face Hub without leaving the launcher.
  • Load any repo by ID or huggingface.co URL (tree/ / blob/ / resolve/ links auto-seed the revision), then pick a branch or tag — downloads pin to the exact commit SHA so a moving ref can't cause a mismatched download.
  • File browser with human-readable sizes and multi-select, plus Select defaults (smartly picks GGUF files and all shards of a sharded set), Select all, and include/ignore glob patterns (e.g. *.gguf) for quant filtering.
  • Two modes: download selected files or a full repo snapshot, into one or more of your configured model directories.
  • Gated/private repos: optional masked auth token, passed via HF_TOKEN to the download subprocess and never written to disk.
  • Options & progress: Force download, Local files only, tunable Max workers (1–32), live status/progress bar with tqdm-style throughput/ETA, and a reliable Cancel. Backed by the managed venv's huggingface_hub, with a one-click install/update button.

📸 View Environment Variables Screenshot

• Environment Variable Management:
  • Easily enable/disable common CUDA environment variables (e.g., GGML_CUDA_FORCE_MMQ).
  • Add and manage custom environment variables to fine tune CUDA performance.

📸 View Configuration Management Screenshot

• Configuration Hub:
  • Save, load, and delete named launch configurations.
  • Import and export configurations to JSON for sharing or backup.
  • Application settings (last used paths, UI preferences) are remembered.

📸 View Settings Screenshot

• Settings & Appearance:
  • Theme and font-size controls, plus Windows high-DPI scaling.
  • Manage a launcher-owned Python virtual environment: create it, and install / update / remove requests, torch, psutil, and huggingface_hub per venv (shared by the Build and HF Downloader tabs).
• Script Generation:
  • Generate ready-to-use PowerShell (.ps1) and Bash (.sh) scripts from your current settings (including environment variables).
• Cross-Platform Design:
  • Works on Windows (tested), Linux (tested), and macOS (untested).
  • Includes platform-specific considerations for venv activation (for GPU recognition) and terminal launching.
• Dependency Awareness:
  • Checks for optional but recommended dependencies for GPU detection and model information

🆕 Recent Features

• July 2026 — Build (beta) tab: build llama.cpp / ik_llama from source with toolchain auto-detection, CUDA architecture picker, a full backend-aware CMake-flag editor, an "optimized for this system" preset, live cmake preview, in-app streaming builds, and saved build profiles. New HF Downloader tab for browsing and pulling models from the Hugging Face Hub (revision-pinned downloads, GGUF/shard-aware selection, gated-repo token, snapshot mode). Managed Python venv controls added to the Settings tab.
• May 2026 — MTP / Speculative Decoding tab with full cross-backend support: draft GGUF picker, spec-type-aware default prefill, smart draft GPU controls, and auto --parallel 1 for MTP. Reasoning / Thinking controls added to the Chat Template tab (--reasoning, --reasoning-format, --reasoning-budget, --chat-template-kwargs) plus a --jinja toggle.
• April 2026 — Settings tab (theme/font controls, Windows high-DPI); user-orderable GPU list and mmproj selector dropdown; modular project layout (modules/, config/, launchers/); automated test suite + CI workflow.
• January 2026 — --fit (auto-fit context to VRAM) and --parallel slot options; improved GGUF parser.
• December 2025 — GPU layer override; Flash Attention updated for newest llama.cpp API; --cpu-moe works with ik_llama.
• August 2025 — MoE offload options (--cpu-moe, llama.cpp PR 15077); mmproj command-line wiring; checkbox to toggle mmproj scanning for faster directory loads.
• July 2025 — Q6_K KV cache option; model search box and configuration name/handling improvements.
• June 2025 — ik_llama backend with dedicated parameters tab.

📋 Dependencies

Required

• Python 3.10+ with tkinter support (typically included with Python). CI tests on 3.10 / 3.11 / 3.12.
• llama.cpp or ik_llama built with server support (llama-server executable from either backend)
• requests - Required for version checking and updates
  • Install with: pip install requests

Optional (Recommended)

• PyTorch (torch) - Required if you want automatic GPU detection and selection
  • Install in your virtual environment: pip install torch
  • Without PyTorch, you can still manually configure GPU settings
  • Enables automatic CUDA device detection and system resource information
• psutil - Optional for enhanced system information
  • Provides detailed CPU and RAM information across platforms
  • Install with: pip install psutil

Installation Example

# Create and activate virtual environment
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Option 1: Install using requirements.txt (recommended)
pip install -r requirements.txt

# Option 2: Install dependencies individually
pip install requests torch psutil

🛠️ Installation & Setup

1. Clone the Launcher

git clone https://github.com/thad0ctor/llama-server-launcher.git
cd llama-server-launcher

2. Setup Dependencies

Install the required Python dependencies using the provided requirements.txt file:

pip install -r requirements.txt

Or follow the Dependencies section above to install dependencies individually.

3. Build llama.cpp with CUDA Support

You'll need to build llama.cpp or ik_llama separately and point the launcher to the build directory. Here's an example build configuration:

⚠️ Example Environment Disclaimer:
The following build example was tested on Ubuntu 24.04 with CUDA 12.9 and GCC 13. Your build flags may need adjustment based on your system configuration, CUDA version, GCC version, and GPU architecture.

# Navigate to your llama.cpp (or ik_llama) directory
cd /path/to/llama.cpp

# Clean previous builds
rm -rf build CMakeCache.txt CMakeFiles
mkdir build && cd build

# Configure with CUDA support and optimization flags
CC=/usr/bin/gcc-13 CXX=/usr/bin/g++-13 cmake .. \
  -DGGML_CUDA=on \
  -DGGML_CUDA_FORCE_MMQ=on \
  -DCMAKE_CUDA_ARCHITECTURES=120 \
  -DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc \
  -DCMAKE_CUDA_FLAGS="--use_fast_math"

# Build with all available cores
make -j$(nproc)

📚 Need More Build Help?
For additional building guidance, platform-specific instructions, and troubleshooting, refer to the official llama.cpp documentation.

Key Build Flags Explained:

• -DGGML_CUDA=on - Enables CUDA support
• -DGGML_CUDA_FORCE_MMQ=on - Forces use of multi-matrix quantization for better performance
• -DCMAKE_CUDA_ARCHITECTURES=120 - Targets specific GPU architecture (adjust for your GPU)
• -DCMAKE_CUDA_FLAGS="--use_fast_math" - Enables fast math optimizations

4. Configure the Launcher

1. Run the launcher: python llamacpp-server-launcher.py
2. In the Main tab, set the "LLaMa.cpp Root Directory" (or "ik_llama Root Directory" when the ik_llama backend is selected) to your build folder
3. The launcher will automatically find the llama-server executable

 

🚀 Core Components

This launcher aims to streamline your llama.cpp server workflow when working with and testing multiple models while making it more accessible and efficient for both new and experienced users.