Skip to content

Releases

HomeBred-LLM v1.2.1

Choose a tag to compare

@Eisenmann Eisenmann released this 09 Oct 16:59
92f837b

Fixes

Hugging Face downloads failed with 404. The app requested a model folder as if it were a file. It now downloads all files of the chosen variant, plus the config and tokenizer files the model needs.
Failed downloads left a stuck entry. A download that failed stayed “Downloading” in the library after a restart. It is now saved as Error.
Misleading “No GPU” with a GPU installed. The Calculator and the Config page offered a VRAM budget for a card that official builds can’t use, because only the CPU backend ships. The pages now say the GPU was detected but the backend is CPU-only, and the VRAM budget is disabled until a GPU backend is active. A VRAM budget you saved earlier is kept.
Binding exception in the Model Library. The file-size column raised a NotSupportedException from a converter. It was only a logged warning at runtime, but it stopped the debugger. Fixed.

Also in this release: GPU setup help

Compute switch (Calculator → Compute): choose CPU or GPU. With GPU selected, the app checks your NVIDIA driver, the Vulkan loader and the llama.cpp GPU build, and tells you what is missing.
One-click setup: a Set up GPU backend automatically button downloads the official llama.cpp b8816 build for your GPU from GitHub and installs it for your user account. Then restart the app. The release download stays CPU-only and small.
Manual steps: docs/gpu-setup.md has them.
Platforms: the automatic setup covers Windows (CUDA, Vulkan) and Linux (Vulkan). I haven’t tested it against a real GPU yet.

For contributors

AGENTS.md: a build, run and test guide.
UiSmokeTests: a headless UI check that saves screenshots.
HOMEBRED_DATA_DIR: an environment variable that moves all app data.

Upgrade notes

Nothing to do; the database is upgraded automatically.
Speed and fit numbers remain estimates.

What's Changed

  • Fix NotSupportedException from BytesToReadableConverter.ConvertBack by @Eisenmann in #6

Full Changelog: v1.2.0...v1.2.1

v1.2.0

Choose a tag to compare

@Eisenmann Eisenmann released this 09 Oct 11:22
4aecae6

HomeBred-LLM v1.2.0 — Run models bigger than your GPU

This release lets HomeBred-LLM run GGUF models that don't fit in VRAM by
splitting them across VRAM → RAM → disk. A new Capacity Calculator
tells you what your machine can run before you download anything.

✨ Highlights

🧠 Tiered memory for GGUF models

  • Per-model memory budgets (Config → Memory tiers): set how much VRAM and RAM
    a model may use, or leave both on Auto, and choose whether weights may stay on disk.
  • Smart placement: weights read on every token go to VRAM first. Experts and
    layers that don't fit are computed on the CPU from RAM, and anything beyond
    the RAM budget is read from disk on demand.
  • MoE-aware RAM tier: for Mixture-of-Experts models, the app keeps the
    individual experts the model actually uses in RAM. You can optionally lock
    them so the OS can't page them out.
  • Learns expert usage: while the model is idle, the app records which
    experts each layer routes to. It then re-targets the RAM tier live, with no
    reload, and saves the profile for the next start.
  • Plan preview before loading: see the VRAM/RAM/disk split, an estimated
    tok/s, which part (GPU, CPU or disk) slows generation most, plus warnings
    and suggestions.
  • KV cache type (F16 / Q8_0 / Q4_0), KV placement and flash attention are now
    configurable per model.

🧮 Capacity Calculator (new page)

  • "How big a model can I run?" for three strategies: GPU only,
    GPU + RAM, and GPU + RAM + disk. Works for dense and MoE models, with
    an optional minimum tok/s.
  • Estimated-speed-vs-model-size chart.
  • "Can I run this?" for any local model, with suggestions: the largest
    context that keeps everything on the GPU, a quant that would fit, a Q8_0 KV
    cache, or how much more RAM to allot.
  • Hardware benchmark: measures RAM bandwidth and the read speed of your
    model drive. Estimates also self-calibrate from your real runs.

🏷️ Fit badges in the Model Library

Every local model and Hugging Face file shows whether it fits on your GPU,
needs RAM offload, needs the disk tier, or is too large, with an estimated
speed.

📊 Analytics

  • Live tier view for the running model.
  • Prefill and decode speed reported separately, next to the estimate.
  • Disk reads, PCIe traffic and page faults.
  • Modeled time per token, split by where it's spent.
  • MoE expert-routing heatmap.

🧪 Experimental: per-expert VRAM cache (Phase 2)

  • Keeps the hottest individual experts of each MoE layer in VRAM slots; the
    rest run on the CPU. Output is bit-identical to normal inference.
  • Requires a patched llama.cpp built from native/llama.cpp-hbec
    (build.ps1 -Backend cuda / build.sh cuda). The app loads it
    automatically from HomeBred-LLM/native/ when present.
  • Opt-out per model: Config → Memory tiers → Per-expert VRAM cache.
  • ⚠️ The CUDA build has not been validated on GPU hardware yet; run the
    bundled hbec_test after building.

🔧 Changes & fixes

  • New GGUF models default to Tiered placement with Auto budgets.
    Simple mode (GPU Layer Count only) is still available per model.
  • Inference stats are now recorded once per chat turn instead of being
    repeated in every 5-second sample, so historical averages are more accurate.
  • Fixed analytics/calculator charts crashing when an axis had no data, and
    made chart legends readable in the dark theme.
  • Config page: Save/Reset moved below all settings.

⬆️ Upgrading

  • The database is upgraded automatically on first launch (new tables and
    analytics columns). No action needed.
  • GPU note: official builds ship with the CPU llama.cpp backend
    (LLamaSharp.Backend.Cpu), so the VRAM tier is disabled and the app says
    so. For GPU offload, reference LLamaSharp.Backend.Cuda12 or use the
    patched native build above.

🛠️ For developers

  • New TieringVerificationTests console runner covering estimators, the
    planner, the calculator, the warm tier, schema upgrade, and end-to-end
    native tests on a tiny MoE test model.
  • HomeBred-LLM/TestData/test_tiny_moe.gguf is git-ignored by *.gguf;
    add it with git add -f.
  • Docs: native/llama.cpp-hbec/README.md, docs/hb_expertcache.h.

What's Changed

  • ci: added build pipeline by @Eisenmann in #4
  • Run big LLMs on the local GPU — tiered memory + per-expert VRAM cache by @Eisenmann in #5

Full Changelog: 1.1.0...v1.2.0

v1.1.0

Choose a tag to compare

@Eisenmann Eisenmann released this 20 Sep 17:55
3955462

🏠 HomeBred-LLM — Local Model Files Support

This release adds the ability to load and run local model files directly from disk, with no downloads or external services required. Pick a .gguf file or an ONNX export from your own machine, point HomeBred-LLM at it, and start chatting immediately.

✨ New Features

🧠 GGUF Local Model Support

  • New GgufReader / GgufMetadataReader services parse GGUF v2/v3 files in-place to extract metadata (name, architecture, quantization, parameter count).
  • llama.cpp inference engine (LlamaCppInferenceService) with native log capture via NativeLogBuffer — replaces the older LLamaSharp service and runs entirely in-process.
  • New GgufVerificationTests console project that validates GGUF reading against real files.

🟦 ONNX Local Model Support

  • OnnxRuntimeService and OnnxModelMetadataReader load and run local ONNX model exports.
  • Metadata extraction for architecture, parameter count, and quantization from ONNX files.

📂 Local Model Import

  • New LocalModelImportService registers a local GGUF or ONNX model in the Model Library — the file is used in-place (no copying, since model files are often many gigabytes).
  • Import-time diagnostics surface warnings (e.g., HuggingFace-style tensor names, vocab size mismatch) as non-blocking badges in the UI.
  • InferenceServiceRouter automatically picks the correct runtime based on file type: GGUF → llama.cpp, ONNX → ONNX Runtime.

🔧 LoRA & UI Improvements

  • Reworked LoraImportService for the new local-model workflow.
  • Model Library and Config views updated with a "Add Local Model" flow; app data layer (EF Core AppDbContext, LocalModel) extended to persist local models across sessions.

🛠 What Changed

  • 26 files changed, +2,594 / −431 lines
  • New inference router, GGUF reader, ONNX reader, llama.cpp service, local model import, and verification tests
  • UI/ViewModel updates for local model browsing and import

v1.0.0

Choose a tag to compare

@Eisenmann Eisenmann released this 01 Jul 19:06
d122f69

Release Notes — HomeBred-LLM v1.0.0
First public release · 2026-07-01

Cross-platform, self-hosted LLM desktop app — download, configure, run, and chat with local models entirely on your own machine. No Ollama, no Docker, no Python, no cloud. Everything runs in-process on Windows, Linux, and macOS.

🎉 Highlights
HomeBred-LLM embeds llama.cpp in-process via LLamaSharp — the same engine that powers Ollama — with model management, GPU monitoring, chat, a local API, and LoRA adapters all inside one self-contained Avalonia desktop app.

✨ Features
📦 Model Library
Search HuggingFace directly from the app — no HF CLI or Python.
Browse a repo's GGUF files by quantization and size.
Download with a real-time progress bar (MB downloaded / total).
Automatic vision projector (mmproj) detection — multimodal models get their projector downloaded alongside the weights.
⚙️ Model Configuration
Per-model tuning of:

Temperature, Top-P, Top-K, repeat penalty
Context size, GPU layer count, CPU threads, batch size
Max output tokens and system prompt
Save / reset-to-defaults, persisted in the local database.
🧬 LoRA Adapters (specialize without retraining)
Attach one or more pre-trained GGUF LoRA adapters to any model.
Per-adapter blend scale and on/off toggle; stack multiple adapters.
Applied to both chat and the embeddings endpoint at load time.
Imported files are validated (GGUF check) and stored in a managed adapters/ folder.
Application only — training is done externally and converted to GGUF.
▶️ Model Lifecycle
Start — load weights into GPU VRAM (or CPU).
Stop — unload without restarting the app.
Remove — delete the file and its database record.
💬 Streaming Chat
Token-by-token streaming output.
Multiple named chat sessions per model.
Image and text file attachments (drag-and-drop or picker); images route to vision-capable models.
Per-message stats: tokens/sec and latency.
📊 Live Analytics
Sampled every 5 seconds and charted:

GPU utilization %, VRAM used/total, GPU temperature (via NVML)
CPU utilization and RAM
Per-request tokens/sec, time-to-first-token, total inference time, and prompt/output token counts
Cleanup tool — delete metrics for any model over any date range without touching the model.
🔌 Local API Endpoint
Optional, opt-in per model.
OpenAI-compatible POST /v1/chat/completions (streaming supported) and POST /v1/embeddings.
Configurable port, bound to 127.0.0.1 only — no separate process.
🖥️ Platform & engine
.NET 10, cross-platform: Windows · Linux · macOS.
GPU: NVIDIA CUDA 12 with automatic CPU fallback — both backends bundled; CUDA selected automatically when drivers are present.
Inference: LLamaSharp 0.27 (P/Invoke bindings to llama.cpp), fully in-process.
UI: Avalonia 11.2 + CommunityToolkit.Mvvm; LiveCharts2 for analytics.
Storage: SQLite via EF Core — auto-created on first launch, schema self-upgrades in place.
📁 Data storage
All data lives in one folder per platform (%LOCALAPPDATA%\HomeBred-LLM\ on Windows, ~/.local/share/HomeBred-LLM/ on Linux, ~/Library/Application Support/HomeBred-LLM/ on macOS):

homebred.db # models, configs, analytics, chat history
models/ # downloaded GGUF weights
adapters/ # imported LoRA adapters
attachments/ # chat file attachments
Uninstall = delete the binary and this folder. No registry entries, no system-wide changes.

✅ Supported models
Any GGUF on HuggingFace, including LLaMA 3.x, Mistral / Mixtral, Qwen 2/2.5, Phi-3/Phi-4, Gemma 2/3, DeepSeek, and any other llama.cpp-supported architecture. Recommended quantizations: Q4_K_M, Q5_K_M, Q6_K.

📋 Requirements
.NET 10 runtime (or publish self-contained)
4 GB VRAM minimum for small models; 8–16 GB for 13B+ at full GPU offload
Disk: ~2 GB (Q4 7B) to ~40 GB (Q4 70B) per model
⚠️ Known limitations
LoRA adapters apply at load time — Stop/Start a model to pick up changes.
Only GGUF-format adapters are supported (convert PEFT/.safetensors externally).
Apple Silicon / Metal and AMD ROCm backends are planned, not yet included.