Repository navigation
Releases
v0.1.2 — local-model chat on Linux, private model ports, installer upgrades
Chat with local models works on Linux, model ports are no longer public, and re-running the installer now really upgrades.
Install or upgrade
curl -fsSL https://raw.githubusercontent.com/LLMPanel/llmpanel-oss/master/install.sh | sudo bashUpgrade by re-running the installer. This release changes docker-compose.yml, and docker compose pull alone does not pick that up. The installer now refreshes the compose files and keeps your secrets, settings and data. Images: ghcr.io/llmpanel/llmpanel-api and ghcr.io/llmpanel/llmpanel-web, tags 0.1.2, 0.1 and latest.
After upgrading, redeploy any local model created before 0.1.2 to close its public port (see below).
Fixes
Chat with local models failed on Linux. Sending a message to a Local Docker deployment returned Could not connect to the model at http://host.docker.internal:39000: fetch failed. The chat is streamed by the web container, which could not resolve host.docker.internal on Linux. It now can.
Local models were reachable from the internet without an API key. Model containers published their port (39000 and up) on every interface, so anyone could call http://<your-server>:39000/v1/... directly, bypassing API keys. New deployments join the panel's own Docker network and publish no port at all; the gateway reaches them by container name and still checks API keys. Deployments created by earlier releases keep their old port until you redeploy them.
Re-running the installer did not upgrade everything. It kept compose files already on disk, so compose fixes never arrived, and it rewrote .env from scratch, blanking settings that were not passed again (a domain install fell back to plain HTTP). Now each run:
- re-downloads the compose files, saving a changed copy as
docker-compose.yml.bak.<time>(LLMPANEL_KEEP_FILES=1keeps the files on disk as they are); - keeps every setting in
.envunless you pass a new value, including keys the installer does not write itself, such asDATABASE_URL.
The documented domain install did not pass its settings. In PANEL_DOMAIN=... curl ... | sudo -E bash the variables reach only curl, so the install silently used plain HTTP. Pass them after sudo instead:
curl -fsSL https://raw.githubusercontent.com/LLMPanel/llmpanel-oss/master/install.sh | sudo \
PANEL_DOMAIN=panel.example.com \
ACME_EMAIL=you@example.com \
bashImproved
Reasoning folds away in chat. Thinking models (Qwen3, DeepSeek-R1 and similar) stream their reasoning as <think>…</think>, which used to fill the reply bubble. It now sits in a collapsed Thought process toggle above the answer, labelled Thinking… while it streams. If a reply runs out of tokens while still thinking, the chat says to raise Max tokens. Copy copies the answer only.
v0.1.1 — run models on the CPU when there's no GPU
Local Docker now runs models on the CPU when the host has no GPU.
Install or upgrade
curl -fsSL https://raw.githubusercontent.com/LLMPanel/llmpanel-oss/master/install.sh | sudo bashRe-running the installer upgrades an existing install and keeps its secrets. You can also run cd /data/llmpanel && docker compose pull && docker compose up -d. Images: ghcr.io/llmpanel/llmpanel-api and ghcr.io/llmpanel/llmpanel-web, tags 0.1.1, 0.1 and latest.
What's new
CPU inference on hosts without a GPU. Before, a CPU-only server stopped at "No GPU detected — cloud providers only" in the deploy wizard. Now Local Docker serves GGUF models on the CPU with llama.cpp (ghcr.io/ggml-org/llama.cpp:server-b11515).
- Step 3 of the wizard becomes CPU & RAM. It shows the host's cores and free RAM, and whether the model fits.
- Models with a CPU build: Qwen3.5 0.8B, Phi-4-mini, Qwen3 8B and 14B, Gemma 3 12B, gpt-oss 20B, Devstral Small and Mistral Small 24B (Q4_K_M weights). The rest are shown but can't be picked.
- The API also refuses a model with no CPU build, or one that needs more RAM than is free.
- Downloaded weights are kept in a Docker volume, so a restart doesn't fetch them again.
- Metrics show container CPU, memory and uptime instead of GPU telemetry.
CPU inference is much slower than a GPU, so small models are the practical choice. For GPU speed, add a RunPod or Vast.ai key under Providers.
CPU mode turns on when the installer finds no usable NVIDIA GPU (GPU_AVAILABLE=false in /data/llmpanel/.env). Hosts with a GPU keep using vLLM as before.
v0.1.0 — first public release
The first tagged release of LLMPanel, an open-source control panel for self-hosted LLM inference.
Install
curl -fsSL https://raw.githubusercontent.com/LLMPanel/llmpanel-oss/master/install.sh | sudo bashThis pulls ghcr.io/llmpanel/llmpanel-api and ghcr.io/llmpanel/llmpanel-web (tags 0.1.0, 0.1 and latest). See the README for HTTPS on your own domain, configuration, and upgrading.
What's in it
- Deploy models from a registry onto Local Docker (NVIDIA GPUs on the host), RunPod or Vast.ai, served by vLLM.
- An OpenAI-compatible gateway at
/v1with scoped, rate-limited API keys. - A chat playground, usage analytics, request and container logs, and per-GPU metrics.
- Image generation, plus knowledge bases, agents and chat channels (beta).
- Alerts to Telegram, Discord or Slack; per-deployment backups; workspaces with roles and 2FA.
Read before exposing it
Registration stays open after setup, and Local Docker model ports serve without an API key — see Before you expose it to the internet.