v1.15.0

Added

• LiteRT-LM in the standalone Android app. The app now ships LiteRT-LM built in, a native port of litert-lm serve
  over the same runtime Termux users run through Python. It needs no Python, updates with the app, and shows up in the
  app's Engines screen beside the bundled llama.cpp engine. Contributed by @ciroprogamer (#281).
• Download LiteRT-LM models from Discover. A LiteRT-LM repo (e.g. litert-community/…) lists its .litertlm
  bundles in the repo view, a chosen bundle downloads as one file, a direct .litertlm link imports like a .gguf
  one, and model search is no longer limited to GGUF repos while LiteRT-LM is the active engine. Contributed by
  @ciroprogamer (#277).
• Pause and resume downloads (Models → Discover → Downloads). Pause a running or queued download, or use Pause all
  and Resume all. A paused download keeps what it has, also across a restart, and Resume continues from there. Cancel
  all leaves paused downloads alone; a paused or failed download's Discard deletes its partial file, and Cancel or
  Discard asks first above 50 MB. The Library shows one line while downloads are running, paused or failed ("2
  downloads running · 1 paused · 1 failed") with a Show link. Asking again for a file whose download is paused
  (Discover, Import from URL, onboarding) resumes it and says "Resumed" or "Already downloading".
• Install a prebuilt engine from a GitHub release (Engines → Add engine → From a GitHub release, or a card's
  Install prebuilt). Name a repo, or pick a known fork, and TurboLLM lists the newest release's builds for your OS, CPU
  and GPU, downloads the one you choose, checks it runs, and only then adds it. Release engines update from new tags
  with the same check-before-swap, and the install keeps going if you close the dialog, leave the screen or reload;
  opening it again shows its progress. Cancel stops it.
• Default KV cache types (Settings → Models & loading → Advanced, #288). Pick the K and V cache types a model
  loads with when it has none saved of its own: q8_0 roughly halves KV memory, q4_0 quarters it. Automatic keeps
  today's choice (f16, q4_0 on Android). The list is the selected engine's; a model on an engine without the chosen
  type runs at f16. Contributed by @shunya0 (#296).
• Hide a model (Models → Library, a model's ⋯ menu → Hide). It leaves the Library list and every model picker
  until you unhide it (Show hidden models under the list). A hidden model keeps its files and settings, still loads
  when something names it (an API client, a routine, turbollm launch --model), and the choice holds on every device.
• Show models for every engine (Settings → Models & loading, off by default) lists every model in the Models
  screen, the pickers and the gateway's GET /v1/models, also those the selected engine is not expected to run. A
  model still loads only on the selected engine; if that engine can't load it, you see the engine's own error.
• Search in the "Load a model" picker (Chat, the Code composer and the terminal toolbar). Type to filter by name,
  quant, machine or repo; arrow keys and Enter pick a model. Clicking a model now closes the picker.
• Tool-call approvals in chat wait a set time (Settings → Tool permissions → Wait for an answer, 30 minutes by
  default, 0 waits forever). A call set to Ask that gets no answer in time is not run, the chat says so, and the reply
  goes on as after a Deny. Code's ask mode still waits until you answer.
• API additions for apps and scripts. GET /api/v1/conversations/:id reports runningReplyId and compacting,
  and each row of GET /api/v1/conversations reports running. GET /api/v1/conversations/:id/stream follows a
  chat's running reply from any page or device, and POST /api/v1/chat/stop takes the messageId of the reply to
  stop. GET /api/v1/activity now covers model loads, auto-tune, skill writing, title and memory calls, approval waits
  (in waiting), downloads and a status summary. GET /api/v1/status reports poolModels and serving. POST /api/v1/daemon/flush writes the database to disk at once. GET /api/v1/code/sessions/:id reports compacting, and
  each agent of GET /api/v1/code/agents reports installing.

Changed

• API keys are now required in more places when TurboLLM is reachable from other devices. If you run TurboLLM
  on your LAN, behind Tailscale Serve or Cloudflare Access, or in Docker, check this one:
  • --addr 0.0.0.0 (or any non-loopback daemon.host) now makes LAN clients present the API key, as --help always
    said. Before, that bind had no key check at all unless LAN access was also switched on in Settings.
  • A keyless device on an open LAN, a Tailscale Serve identity or a Cloudflare Access login now needs a real API key
    to delete a model, download from a URL, add or choose a model folder, start an engine with its own flags or model
    paths, add, edit or delete an MCP server, use MCP tools in a chat, agent or routine, save a skill, purge an engine
    or delete a build, or install the ComfyUI gate node. The web app asks for the key when this happens.
  • A remote-access token without "Load models" can no longer swap in a model that isn't loaded (it gets a 503); it
    still uses the models that are loaded. A remote-access token can now chat from the web UI.
  • Browser writes to /api/* from another website, or another localhost port, get a 403, and the UI can no longer be
    shown inside a frame. A reverse proxy on the same machine that forwards to 127.0.0.1 under its own public host name
    is no longer trusted as local; give it a key.
  • On Android, other apps on the phone are treated like LAN devices and need a key.
  • config.json and its backup are readable by your user only, and saved MCP environment values are no longer sent
    back to the browser (the edit form shows them masked).
• Docker: the Compose files use host networking on Linux. docker-compose-nvidia.yaml and
  docker-compose-amd.yaml now set network_mode: "host" instead of publishing port 6996, so the host's browser
  counts as local (the workspace and folder browsers work), and they mount ~/workspace into the container for Code.
  The AMD image adds Vulkan packages and an HSA_OVERRIDE_GFX_VERSION setting for consumer GPUs. The images run
  tini as PID 1, and each container now mints its own API key on first start (the image build used to bake one key
  into every container). Contributed by @ahmadteeb (#292).
• A chat reply runs in the daemon, not in the page. Closing, reloading or leaving the page, Android's Back or a
  dropped connection no longer stops a reply, a Regenerate or a Save & Resend, or their tool calls and approvals. A
  page that opens the chat again, on this device or another, shows the reply live with its Stop button. Stop, Eject,
  deleting the chat, stopping the engine, and losing LAN or remote access still stop it, and so does revoking the key
  the reply was sent with. The text-classification playground's runs work the same way and gain a Cancel button.
• A model's Load settings save only what you changed. Load, Save, Remember these settings, Save as… and Auto-tune
  used to save the whole panel, so the first save froze every default (context 8192, GPU layers 99, the default
  sampling) and later changes to Settings → Model Defaults never reached that model again. Now only the controls you
  moved (plus what Auto-tune measured) are saved. Older saved values that equal the defaults count as not set.
• Desktop app: closing the window keeps TurboLLM running on Windows and Linux, with a tray icon (Open TurboLLM,
  Quit TurboLLM), so replies, Code runs, downloads and auto-tunes go on. Quit from the tray or the app menu stops it as
  before and installs a downloaded update. On a Linux desktop with no tray icons, closing minimizes instead. The app
  no longer opens a browser tab on launch, opens links in your system browser instead of an in-app window, runs as a
  single instance, and starts on the port saved in Settings.
• LiteRT-LM models use their own defaults, and your settings reach them. Backend Auto now uses the backend the
  bundle declares; the context is the bundle's own unless you set one; the bundle's own sampling is shown and used,
  and the sampling, penalties and stop strings you set are sent with each request. Thinking controls (chat, Code,
  Claude Code) now reach .litertlm models, and a bundle with a vision or audio encoder is treated as a vision or
  audio model. Models whose template supports tools now get the chat's tools (web search, fetch, MCP, skills) when
  they fit in the context, and past reasoning is sent the way the model's template expects. Top K 0 means "no limit",
  and Min P is no longer offered (LiteRT-LM has none). The chat says when a setting can't be applied.
• LiteRT-LM through the gateway: tool_choice: "none" sends no tools, a forced tool call ("required" or a named
  function) gets a 400 because LiteRT-LM can't force one, and parameters litert-lm serve ignores (min_p, stop,
  penalties) are named in an X-Unapplied-Parameters response header. An Anthropic client's tool_choice: {"type": "none"} is now "none" on every engine.
• Gateway: GET /v1/models (what Claude Code's /model shows) leaves out models the selected engine can't run,
  unless Show models for every engine is on. Thinking "Off" in a Code terminal session now really turns thinking off
  (it used to remove the cap), and /v1/messages honours thinking: {"type": "disabled"}. Streaming OpenAI clients
  get the usage chunk only when they ask for it, and a stream the engine cut off ends with an error instead of a
  normal finish.
• One TurboLLM per data folder. A second turbollm on the same data folder is refused, even on another port (it
  used to run alongside and share the database). Give it its own --config if you really want two.
• A Settings restart started from a terminal stays in that terminal. Ctrl+C, docker stop, turbollm --stop or
  closing the terminal still ends it, and the new daemon's output (a fresh --print-token included) appears there;
  restart.log is no longer written. A restart stops running Code turns and pauses running downloads.
• Code: what an agent's bash command starts in the background keeps running until its session is archived or
  deleted, or TurboLLM stops. Closing a Code terminal, or its shell exiting, ends everything started in it, nohup
  jobs included. A repository's own pi extensions (.pi/extensions) no longer load, since that ran the repo's code
  inside TurboLLM with no approval. ssh no longer asks for a passphrase or host key in TurboLLM's terminal during a
  git push; it uses the desktop's askpass dialog or fails at once.
• Android app: it reports its GPU to TurboLLM directly, keeps the model loaded and the phone awake during any
  engine work (loads, auto-tune, skill writing, a reply's title), keeps running while LAN, remote access or Turbo Link
  is on, and no longer checks npm for updates (it updates through the app).

Fixed

• Nothing TurboLLM starts outlives it. Engines and their workers, extra loaded models, tunnels and the
  tailscale CLI, routine runs, agent CLI installs, MCP and language servers, engine builds and installs, probes, the
  export browser and the Code agent's searches now stop when TurboLLM stops, by any means, including a crash or a
  kill. Some used to run on forever. On Windows, their own children end too. git gets a moment to drop its locks, so
  .git/index.lock no longer blocks the next command.
• TurboLLM shuts down gracefully when you close the terminal it runs in, press Ctrl+C (or Ctrl+Break on Windows)
  under npm start, or Quit the desktop app on Windows; all of these used to end it at once. If the desktop app
  crashes, its daemon shuts down by itself, and reopening the app right after a Quit waits for the old daemon. A
  Settings restart under npm start works again.
• Chat replies are saved correctly when they're cut short. A Stop, or closing the page, saved the reply as
  finished (and still ran a tool call it was streaming); it is now saved as stopped and no tool call starts. A reply
  running when TurboLLM stops or restarts is saved as stopped with the text it had, and a reply's text is saved every
  10 seconds while it streams, so a crash keeps most of it. An empty cut-off reply reads "Generation failed or was
  stopped." A reload keeps the chat that was open.
• Android: chats are no longer lost when the app is closed. Chats are written to disk about a second after they
  change instead of every 10 seconds, and on Quit. A kill mid-write can no longer leave a broken database that loses
  every conversation, a clean shutdown now saves at all, and deleting a conversation removes all of its data.
  Contributed by @giveen (#283).
• The idle timeout counts from the model's last use, not from when it was loaded, so a model in active use is no
  longer unloaded between two replies. It also no longer unloads a model during a compaction or a reply's title and
  memory calls.
• TurboQuant installs and updates are reliable. An install whose GPU backend doesn't actually load (for example
  the Windows CUDA 13.3 archive, or an old driver) no longer silently runs on the CPU: TurboLLM checks the devices the
  build sees and moves on to the next build (CUDA 13.3 → CUDA 12.4 → Vulkan → CPU). The update check picks the newest
  release by version and can't offer an older one, a failed download or offline check leaves the working engine in
  place, a failed nvidia-smi no longer swaps a working CUDA build for Vulkan, an update is no longer offered forever
  after it's installed, and downloads are checked against GitHub's published sha256 where one exists. CUDA is detected
  on 600-series NVIDIA drivers. Contributed by @giveen (#284).
• Downloads: an hf.co link now downloads like a huggingface.co one, so gated models work from the short domain
  (contributed by @shunya0, #295). Your Hugging Face token is now sent only to Hugging Face, never to another download
  host. A resumed file that changed on the server starts over (or stops with a clear message for Hugging Face) instead
  of being spliced together, and Hugging Face downloads stay on the version they started from. A .gguf the server cut
  short is no longer marked done. A full disk says "Storage is full" and keeps the partial file. Dismiss all no longer
  deletes a failed download's partial file, Cancel leaves no empty .part behind, a stalled download stops showing
  its last speed, and a server that closes connections no longer crashes TurboLLM. Onboarding finds its download again
  after a closed tab or a restart.
• LiteRT-LM errors say what went wrong. A load failure or a refused request now shows litert-lm serve's own
  reason instead of "fetch failed" or an HTML error page, and errors the engine sends mid-stream reach the chat instead
  of an empty message. Extension runs and routines show the engine's reason on every engine. Writing a skill with no
  model loaded, or one the engine refuses, now says why, and LiteRT-LM models can write skills. The context slider
  stops at the bundle's real limit, read from the file. Install no longer redoes a finished LiteRT-LM environment, and
  does redo a broken one for every Python engine.
• Hardware readings: RAM usage counts the page cache as available, which made Linux and Android machines look
  nearly out of memory. Auto CPU threads use the performance cores on phones and other big.LITTLE arm64 CPUs. On
  Android, the CPU name, core count, CPU usage and GPU load are read correctly.
• A finished auto-tune result survives Eject, loading a model, a restart or Quit; the results dialog offers it the
  next time you open that model. The VRAM estimate no longer counts a non-unified KV cache once per parallel slot.
• Code: Code terminals open on macOS (they failed with "posix_spawnp failed."). An agent terminal still printing
  with no tab open is no longer killed after 5 minutes. A running /compact shows on a reopened page and blocks a new
  turn until it ends. An agent CLI install shows "Installing…" after a reload and only runs once. The agent picker no
  longer hangs when an agent CLI's --version never answers. Committing a large new folder works on Windows, and
  worktree branch names are always ones Git for Windows can create.
• An empty model picker says why ("N hidden: the selected engine can't run them…" or "Download one in Models →
  Discover."), instead of a faint "No models found".
• The web app keeps working while the browser reports itself offline (Wi-Fi off with TurboLLM on the same machine).
• fetch_url can no longer be tricked into reaching TurboLLM's own API or your local network through DNS rebinding
  or a redirect.
• /v1/messages (Claude Code and other Anthropic clients) no longer drops a user turn that holds only a PDF or an
  empty content array; content the model can't read is replaced with a short note so the conversation can go on.
• Extension API: appending a message with generate: false to a chat that is answering is refused with 409
  run_active, like every other change to that chat.
• Desktop app: a port changed in Settings is kept when the app starts again (TURBOLLM_PORT still wins).
• Turbo Link's list of linked machines no longer shows remote-access tokens.
• KoboldCpp starts on the CPU on a machine with no GPU or with GPU layers 0, instead of failing at launch.
• Remote access: ngrok with a reserved custom domain connects instead of timing out.
• Updates: "Update & restart" works for npm installs on Windows, and Termux installs now see and apply updates.
• --addr accepts IPv6 addresses (--addr [::1]:6996), and clients reach a daemon bound to ::1 or another
  loopback address.