v0.4.0 — ComfyUI image generation + model store filtering and bilingual descriptions

What's new

ComfyUI image generation

• Generate images on a remote ComfyUI instance directly from the web UI, with live progress, previews and per-target model listing.
• Handles both weight layouts: a single .safetensors checkpoint, or UNet + CLIP + VAE for Qwen-Image style models. Filenames are matched against what ComfyUI actually reports, so extension-less names no longer fail validation.
• Optional AI prompt enhancement: your text model rewrites the description into a full prompt and suggests steps / CFG. Toggle it off to send your prompt and parameters untouched.

Model store

• Filter by category: text / image / video, with counts and a "runnable on this GPU" filter.
• Cards now show version, parameter count, precision, output resolution, steps and CFG, loader type, and the real weight filenames instead of just a model name.
• Model descriptions are bilingual and follow the UI language.

Deployment backends

• SGLang added as an inference engine (safetensors).
• llama.cpp backend selection by GPU vendor (CUDA / ROCm / Vulkan / Metal), including AMD GPU detection on Linux.
• One-click deployment now lets you pick which backend to run when more than one is available.

Docs

• README covers image generation, the model store and the new backends.

Full changelog: v0.3.0...v0.4.0

v0.3.0 — llama.cpp backend selection + AMD GPU support on Linux

New

• llama.cpp backend selection — install the build matching the machine's GPU: CUDA, ROCm/HIP, Vulkan or CPU. auto decides on its own (NVIDIA → CUDA, AMD/Intel → Vulkan, Apple → Metal), and the installer downloads the matching official asset instead of always the CUDA one; CUDA also pulls the matching cudart runtime.
• AMD/Intel GPUs on Linux — Radeon and Intel cards are now detected in the hardware check and in the live monitor, read from sysfs amdgpu counters. A Radeon machine on Linux used to be reported as having no GPU.

Fixes

• One-click install on NVIDIA machines pulled the CPU build — backend selection reads a vendor field that the nvidia-smi path never filled in, so every NVIDIA machine looked GPU-less and got the CPU package.
• The Windows installer was broken before that too — it asked for releases/latest and matched a hard-coded bin-win-cuda-cu12 asset name, while upstream now ships cuda-12.4 / rocm-10.0 / vulkan / cpu assets and its newest release carries no platform binaries at all.
• Saving a target no longer duplicates it (#1) — the Settings form always started blank, so every save appended a new entry. Targets now match by id first, then by host/port/user/name, and an existing one can be loaded back into the form.
• Failed install steps are reported instead of being silently dropped.

Full Changelog: v0.2.0...v0.3.0

v0.2.0 — SGLang engine support + tuning pipeline localization

New

• SGLang engine support — deploy and monitor SGLang alongside llama.cpp and vLLM, with Prometheus metrics collection and in-tool installation.

Improvements

• Tuning pipeline fully localized — tuning logs and user-facing messages now follow the interface language (backend message registry, 85 call sites). English UI users no longer get Chinese analysis text: AI tuning now instructs the model to write its reasoning in English, while the Chinese prompt body stays untouched.
• AI tuning is the first and default tab on the Tune page, and every backend request carries the current UI language.

Fixes

• Windows GPU detection — cards that expose no standard name (e.g. AMD) are now recognized via WMI, registry qwMemorySize and GPU performance counters (#1).
• Speculative decoding tuning — --spec-draft-p-min joins the continuous grid (0.0 / 0.5 / 0.8), with float parameter parsing fixed (#2).
• Benchmark accuracy — decode throughput was measured over a window that hit EOS after ~24 tokens, reading 30–45% below the real rate; the benchmark now uses ignore_eos, a 512-token window and 2 warmup runs.

Full Changelog: v0.1.1...v0.2.0

v0.1.1 — Deploy page: no more param flash + remembers last model

Fixes

• Deploy page parameter flash: fixed a race condition where tuned (AI/auto-tuner) parameters would briefly show, then get overwritten by hardware-adaptive generated parameters. The param-loading effect now discards stale responses, and a flaky SSH status probe can no longer force the model selection back to the first entry in the list.

Improvements

• Remembers the last selected model per target machine (stored in localStorage). Opening the deploy page now selects, in priority order: the currently running model > the last model you picked > first in the list. Applied to both text (llama.cpp/vLLM) and video (ComfyUI) deploy tabs.

Full Changelog: v0.1.0...v0.1.1

ReadyLLM v0.1.0

First public release of ReadyLLM — a web-based assistant for deploying, monitoring and tuning local LLMs, with ComfyUI image / video generation on top.

Out of the box, ollama often leaves most of a model on the CPU. ReadyLLM finds the parameters that actually fit your hardware and keeps watching it live. Measured on an RTX 4090, same model, same quantization, same context length: peak throughput +37%, average +33%, 100K context +50%.

Highlights

• Unified target management — local machine or remote LAN hosts over SSH (key or password), OS auto-detected, inference engine and model directory configured per target.
• Hardware detection & model recommendation — probes GPU / VRAM / CPU / RAM on the target and recommends models that actually fit.
• Model store — curated built-in GGUF catalog plus dynamic HuggingFace popular-model listing; three download sources (hf-mirror, ModelScope, HuggingFace), background downloads with progress and post-download integrity checks.
• One-click deployment — start / stop llama.cpp and vLLM services with editable runtime parameters and port; missing engines can be installed from within the tool (auto download / compile / pip, then back-fill the path).
• Real-time monitoring — GPU utilization, VRAM, throughput and latency curves pushed live over WebSocket, plus per-day token usage stats (input / output) persisted across backend restarts.
• Smart tuning
  • Auto tuning: two-phase search (coarse over high-impact discrete params, fine over continuous ones) with VRAM feasibility pre-checks and real benchmarking, targeting latency / throughput / prefill.
  • AI Agent tuning: plug in your own LLM API and let it reason out parameters from hardware / model / scenario, feeding benchmark results back for multi-round iteration.
• Image & video generation (ComfyUI) — text-to-video, storyboard generation, long-video per-shot I2V serial generation and stitching, single-image and finished-video upscaling.
• i18n — English by default, switchable to Simplified Chinese.

Requirements

• Python 3.9+
• Node.js 18+
• Target machine has (or can install via this tool) llama.cpp / vLLM / ComfyUI

The control side runs on macOS / Linux / Windows. Note that vLLM does not run natively on Windows — Windows targets must go through WSL2 (the tool shows a guidance prompt).

Quick start

# backend
cd backend && pip install -r requirements.txt
uvicorn app.main:app --host 127.0.0.1 --port 8000

# frontend
cd frontend && npm install && npm run dev

Then open http://localhost:3000, add a target machine, pick a model, hit Tune.

Known limitations in this release

• ComfyUI on the target listens on its own loopback address by default; when the control side cannot reach its port directly, finished-clip preview is proxied through the backend.
• Model sizes are approximate — used only for display and VRAM filtering; the source repository is authoritative.
• Pre-release: the tuning and benchmark paths have been validated on a single NVIDIA consumer GPU (RTX 4090). Other hardware, multi-GPU and AMD are untested.

Full setup, architecture and configuration: README · 简体中文

7 commits since the initial commit (f7d48b2 .. 8587a3d). Source snapshot: tar.gz · zip