Skip to content

Releases: airiclenz/llama-launcher

v1.7.0

Choose a tag to compare

@airiclenz airiclenz released this 14 Aug 15:49

Added

  • A server's API key can come from a command instead of the config file. A servers: entry may now set api_key_cmd — a line whose standard output is the key, typically a lookup in the machine's secret store (security find-generic-password -s llama-launcher -a llamacpp -w). The line goes to the platform shell (sh -c, cmd /C on Windows) so a pipeline works without a wrapper script, and is resolved once at config load for enabled entries only. The command gets no stdin and neither of the launcher's standard streams — it may run under the menu, which owns the terminal — so a store that must ask the human to unlock has to prompt through its own GUI agent. It is bounded by a 60-second timeout and a 64 KiB output cap, and a command that fails, times out, overruns the cap or prints nothing fails the load naming the entry and quoting its stderr: a key source that answers with nothing is a broken source, not a keyless server. Setting both api_key and api_key_cmd, or an api_key_cmd holding only whitespace, also fails the load. The companion marker plaintext_key_ok: true records that a literal key is meant to stay in the file.

  • The interactive menu offers, once per launch, to move a plaintext key into the machine's secret store. Finding enabled entries that still keep api_key as a literal without plaintext_key_ok: true, the launcher raises a single offer before the menu is drawn, naming those entries and their file: move it into the store (the entry reads the key with api_key_cmd afterwards), not now, or never for these entries. Finding none it does nothing at all — not even a store probe. Moving is one ordered move per entry: write the secret, read it back by running the exact api_key_cmd about to be persisted, and rewrite the entry only if what comes back matches, so a failed read-back leaves the config byte-identical and the run keeps working off the literal it has. The rewrite splices text rather than re-marshalling, so comments, blank lines and key order survive; anything it cannot do safely is refused with an edit-by-hand error, and the spliced bytes are re-parsed before an atomic 0600 write. Subcommands never prompt — they emit a one-line warning: on stderr naming the entries, the config file this run read, and the three ways out by hand.

Changed

  • Starting a managed server now checks the target port first and names whoever holds it. The start path refused to spawn where a server of the same backend was still coming up, but had nothing to say about a foreign occupant: it forked, llama-server died inside the grace window with couldn't bind HTTP server socket, and that surfaced as a generic "server exited immediately after start". A pre-flight check now runs between the Starting probe and the binary lookup: if anything is listening on the port, the start is refused (exit code 3) with the port, the address the backend cannot bind, every listening PID with its executable name, and guidance to stop the occupant or configure a different port. The check is one-directional — when lsof cannot answer it reports no occupant and the start proceeds — and deliberately port-wide, since a listener on any interface fails the bind. That asymmetric case also shadows loopback for every other launcher command, so a status that normally answers in under 0.1 s takes 4 s; VS Code's Remote-SSH and Dev Containers port auto-forwarding is a common source, and the README's new "When the port is already taken" section documents the symptom pair.

Security

  • The api_key for llamacpp reaches llama-server through the environment instead of the command line. It used to be appended as --api-key <key>, putting the credential in ps output for every user on the machine for as long as the server ran. It is now exported as LLAMA_API_KEY, which llama-server reads only when no --api-key flag is present, so a user's own extra_args override still wins — and that literal override is still visible in ps, as the README says.

  • The launcher no longer follows redirects from configured LLM servers. authedGet and authedPostJSON each built a default http.Client, which follows up to ten hops — so whatever answers on a configured port could steer an outbound request carrying the launcher's Authorization: Bearer <key> header at a destination of its choosing. Both now share a client whose CheckRedirect returns http.ErrUseLastResponse: the first 3xx comes back unfollowed and no second request is issued.

  • The MCP adapter enables cross-origin protection on its streamable HTTP handler. The IP allowlist admits a whole machine, so a browser page on an allowlisted machine could drive the control plane from a foreign origin. crossOriginHandler now sits between the allowlist and the MCP handler, always on: real MCP clients send no Origin header, or a same-origin one, and pass straight through.

Full details in CHANGELOG.md.

v1.6.3

Choose a tag to compare

@airiclenz airiclenz released this 05 Aug 11:20

Fixed

  • The running Model is shown as a file name instead of a full filesystem path. llama-launcher status printed the absolute --model path in its Model column and on the Active: line, and the interactive menu did the same in its Model: header, the "Active model" pop-up and the unload picker. llama-server reports as its /v1/models id exactly the path it was launched with, and every display site printed it verbatim. A render-time helper, modelDisplayName, now collapses a path-shaped id to its base name at all eight human-facing reads, so the row reads qwen3.6-35B-A3B-Q4_K_M.gguf. Ids that are names rather than paths — LM Studio's qwen/qwen3-8b, Ollama's llama3:8b — are deliberately left alone. Nothing changed at ingest: status --json still emits the raw path in active_model (a documented machine contract, now with its own regression test), and Profile matching plus the ADR-0007 idempotency check keep comparing what the server actually said.

  • The unload picker's no-profile fallback prints with a single pair of parentheses. A loaded model matching no configured Profile rendered as <model> ((no matching profile)); the fallback text is now bare, so the row reads <model> (no matching profile).

Full details in CHANGELOG.md.

v1.6.2

Choose a tag to compare

@airiclenz airiclenz released this 02 Aug 11:38

Added

  • Every Profile list now carries a context-size column, right-aligned between the Profile title and the [server] tag. The value is the effective one (defaults merged with the Profile's own context_size), rendered compactly (512, 4K, 65K, 131K, 1M). All three listing surfaces share one cell computation — the TUI menu, the numbered non-TTY fallback and llama-launcher list; list --json is unchanged. The column appears only when at least one Profile has a value, and a row renders a number only when its LLM Server's ParamSpecs carries the context-size spec, so Ollama rows stay blank. Known limitation: a -c / --ctx-size passed via extra_args is not parsed.

Fixed

  • The frame border is one color for its whole length. Every border glyph now re-asserts normal intensity (\\033[22m) with its color, so the top rail right of a bold title no longer renders bright, and the dashed rule under the title follows the frame's BorderColor instead of a hardcoded dark gray.
  • The framed screens draw a border of uniform weight. The horizontal runs use the light and instead of heavy / , matching the corners, verticals and tees they join. Geometry, width and padding are unchanged.
  • A Profile title containing multibyte characters no longer widens the gap before the interactive menu's context-size, [server] and columns. The label column is measured with visibleWidth (runes) rather than len() (bytes).

v1.6.1

Choose a tag to compare

@airiclenz airiclenz released this 29 Jul 12:02

The library now compiles on macOS, Linux and Windows, and actuates wherever the mechanism exists (ADR-0012).

  • Portability. Platform-specific code moved behind per-OS build tags (process_unix.go / process_windows.go, ui_poll_{darwin,linux,windows}.go), with no runtime.GOOS branching in any shared path. As of 1.6.0 the core built on macOS only, so the facade was unimportable by its own first client's CI. Linux is fully functional; Windows does everything driven over HTTP (discovery, Ollama/LM Studio load-unload, activating a profile against a running server) and refuses unix process control up front rather than attempting it.
  • Two new sentinels: ErrUnsupported and ErrStartupTimeout, re-exported by value so errors.Is matches across the package boundary. A startup timeout is not a failed load — the server is deliberately left running — so clients can now handle "may still come up" without matching on message text.
  • A cross-compile gate: make cross and the make check aggregate build and vet for all three GOOS, so a platform regression fails here instead of in a client.

macOS behaviour is byte-identical to 1.6.0 — this release moves platform-specific code, it does not redesign it.

Shipping as a patch on the 1.6.x line is deliberate: 1.6.x is the library-facade line, and this completes its own promise of importability. Full detail in CHANGELOG.md.

v1.6.0

Choose a tag to compare

@airiclenz airiclenz released this 29 Jul 09:22

llama-launcher is now importable as a Go library (ADR-0011). All launcher code lives in internal/launcher/, which other modules cannot import by Go rule, so driving the launcher from another program meant shelling out to the CLI. A new github.com/airiclenz/llama-launcher/launcher package exports a curated facade: type aliases for the domain types (Config, Profile, ProfileParams, ResolvedProfile, RunningInstance, StopResult, ProgressFunc, NoticeFunc), the two sentinel errors (ErrConfigNotFound, ErrNotRunning — the same values, so errors.Is works across the boundary), and one-line wrappers for the verbs: LoadConfig, DefaultConfigDir, DefaultConfigPath, DiscoverRunningInstances, LoadProfile, Stop, Unload. The documented symbols are the API, and notices are delivered as callbacks rather than written to the host's stderr. The facade adds no dependency to go.mod and leaves the CLI, the MCP adapter, and internal/launcher/'s behaviour untouched.

Also in this release:

  • The two seams that used to print notices straight to stderr — LoadConfig's non-fatal config warnings and the ADR-0007 drift notice — now deliver them to a NoticeFunc sink, the same shape as the existing ProgressFunc. LoadConfigNotify and LoadProfileNotify take the caller's sink, while the exported LoadConfig/LoadProfile keep their signatures and bind the stderr printers they used before, so CLI output is byte-identical to 1.5.0.
  • A shipped loose end is gone from TODO.md: the "stop/unload cannot target a still-loading (503) server" item was resolved by ADR-0010 in 1.5.0.

Usage in the README, the full contract in TDD §16. See CHANGELOG.md for the details.

v1.5.0

Choose a tag to compare

@airiclenz airiclenz released this 19 Jul 18:26

Starting instances are now visible and stoppable (ADR-0010). A server still loading its model (answering 503) used to be invisible to every command; now discovery reports it, status and the interactive menu show it as starting… (with "starting": true in status --json), and stop/unload kill the in-flight load. A plain load onto a Starting address refuses with guidance, while load --restart displaces it. Stop verification no longer reports success when a still-loading server survived the stop.

Also in this release:

  • A build-tagged integration-test layer drives the real backends end-to-end (make test-integration, plus new make test / make test-all targets); the suite bypasses Go's test cache so a green run always means real servers ran.
  • The MCP adapter's server_status, stop_server, tail_log, and unload_model tool descriptions now document the Starting state (no behaviour change — the adapter inherits the new semantics from the CLI).
  • llama-launcher config usage now lists all three subcommands (validate, init, reset).
  • Doc corrections: make install docs no longer claim a ~/.local/bin copy (it points at Homebrew), stale ADR references and line drift fixed.

See CHANGELOG.md for the full details.

v1.4.6

Choose a tag to compare

@airiclenz airiclenz released this 19 Jul 14:58

This section describes what changed relative to the released 1.4.5, which shipped an independent remediation of the same 2026-07-06 review findings. Where both efforts fixed the same defect, the stronger variant was kept; entries below cover only behaviour that differs from 1.4.5 as released.

Changed

  • Profile activation is now testable: the orchestration behind load drives an explicit operations seam. LoadProfile's decision logic — the idempotency/drift check (ADR-0007), the auto_stop_server/auto_unload pass (ADR-0004), and the managed/external activation fork — previously called exec, lsof, process signals, and live HTTP directly, so none of it was covered by tests; only its pure leaf helpers were. The orchestration now runs against a package-private operations interface (ADR-0009) whose production adapter executes the same operations as before, and the first orchestration tests drive that same code against an in-memory fake: the idempotent no-op (with and without a drift notice), --restart, the auto-stop/auto-unload matrix including the foreign-occupant-on-a-shared-address case, and the external model swap/connect paths — without forking a process or opening a socket. The target address is also derived from the resolved profile once and carried through, instead of being re-assembled at each call site. No observable behaviour changes.
  • unload and stop now share one orchestration — the CLI and menu are formatters over it. The "unload on a managed backend means stop the server" rule (ADR-0003/0004) was encoded twice, once in the CLI handler and once in the menu handler, along with duplicated managed-vs-external branching — deleting either handler would have resurrected the logic in the other. Both front ends now call a single Unload/Stop entry point in the server lifecycle layer, built on the activation-operations seam (ADR-0009) and covered by fake-driven tests (managed backend → server stop, external backend → API unload with the server left running). The entry points return a result — the instance acted on, whether the server was stopped or only its model unloaded, and the steps taken — which each front end formats in its own style after the fact. Output text is unchanged; the one visible difference is that stop/unload progress steps now appear once the operation completes instead of streaming live (in the interactive menu, the transient progress popup is replaced by dimmed step lines printed above the outcome).
  • Which parameters the "Show model config" pop-up displays is now owned by each backend, not the menu. The pop-up renderer was the last place outside the backend files that branched on backend names, and it re-encoded each backend's parameter vocabulary by hand — a list that could (and did, before 1.4.5's LM Studio fix) drift from what a load request actually carries. Each LLM Server now exposes a ParamSpecs list — label plus value formatter for exactly the parameters it applies when loading a model or launching a server — and the menu renders that list generically, so a new backend gets a truthful pop-up without any menu change and the compiler forces it to declare its spec. Shared spec definitions keep labels and formatting identical for parameters several backends honour. Two visible corrections ride along: Ollama profiles no longer display context_size — Ollama's load request carries only the model name and a keep-alive, so the pop-up was showing a value the server never received — and llamacpp profiles now list the five sampling parameters 1.4.5 started sending (temperature, repeat_penalty, top_k, top_p, min_p), since each backend's spec shows exactly what it sends.
  • log_retention: 0 now disables cleanup instead of deleting every non-active log. In 1.4.5 a retention of 0 computed a zero-day age threshold, so every timestamped log not belonging to a running server was deleted on each server start. A retention of 0 now means cleanup is disabled (nothing is deleted), same as leaving log_retention unset — only a positive number of days enables age-based deletion. The running-server protection 1.4.5 added to the automatic path is unchanged.
  • status --json entries are now grouped per backend. Each backend's running instances are followed directly by its idle running: false entry, in sorted backend order; 1.4.5 listed all running instances (sorted) first and appended every idle entry after them. The entry set, field names, and exit codes are unchanged.

Fixed

  • The endpoints migration error no longer instructs an impossible move. A config still carrying the pre-1.4 endpoints: section was told "'endpoints' has been merged into 'servers' — move entries to the servers section", but a servers entry only accepts enabled/api_key: a scalar address fails the bool decode, and an addr: key in the mapping form is silently dropped, after which discovery probes the default address and the custom-port instance is never found. The error (and the matching config check problem line) now names the real migration target: a non-default address is set via host/port in the defaults section or on a profile.
  • Bare start with a managed default backend now fails fast instead of forking a doomed server. With only llamacpp enabled, llama-launcher start without --profile forked llama-server with no model flag; the child exited immediately ("--model is required") and the user got an opaque "server exited immediately after start" error plus a log tail. A managed server bakes the Model into its start arguments (ADR-0003), so there is nothing to start without a Profile — the command now fails before forking, with the configuration-error exit code (2) and an actionable message (llamacpp requires a profile: llama-launcher start --profile <name>). External backends (Ollama, LM Studio) still start with no model loaded, as before. The MCP start_server tool description no longer claims that a profile-less start works for every backend.
  • Closing a pop-up now restores the cursor on every exit path — and clears the pop-up box. 1.4.5 re-showed the cursor after the dismissing keypress, but when entering raw mode for that keypress failed the pop-up returned immediately with the cursor still hidden, and no exit path cleared the pop-up from the screen. Every pop-up exit path now clears the pop-up and restores the cursor, including the raw-mode failure path. (The progress pop-ups were already covered: their call sites restore the cursor after each operation.)
  • A health-wait timeout no longer orphans the spawned server silently — and a retry no longer forks a duplicate onto the occupied port. llama-server answers /health with 503 while it loads its model, and a large GGUF on a cold disk can legitimately exceed the launcher's wait window (15 s on start, 30 s on load). On timeout the spawned process kept running, but the error never said so; an immediate retry then saw the address as "unhealthy", forked a second llama-server onto the same port, and reported a misleading "server exited immediately after start" when that duplicate died on the bind. The launcher now deliberately leaves the still-loading server running — killing a legitimately slow model load would be worse — and the timeout error names its PID and log path with recovery guidance (watch llama-launcher logs llamacpp, retry once healthy, or kill <PID>). A "still starting up" server (503) is now distinguished from an unreachable one: every managed start first probes the target address and refuses to spawn a duplicate while an earlier server there is still coming up, naming that server's PID and log path instead. This refusal also applies to load --restart — stop the reported PID first if you really want to replace a loading server. A retry after the server turns healthy behaves as before: the idempotent no-op (ADR-0007). This composes with 1.4.5's start-crash detection: a child that dies within the startup grace window is reaped and reported immediately, so the timeout error can no longer point at an already-dead PID.
  • The stop path no longer signals the same PID twice. StopInstance contradicted its own docstring (which claimed the backend hook ran first) and duplicated EnsureStopped: it signalled the listening PID, then delegated to EnsureStopped, which re-derived and re-signalled the same PID. Both mechanisms now run exactly once, in the documented order (TDD §6.5) — PID signal with the SIGTERM → SIGKILL → port-release escalation, then the backend's native stop hook — inside a single routine; a hook failure surfaces only when the address is still serving afterwards, so it never masks or blocks a stop that already succeeded (ADR-0001: stop is unconditional). Ollama's stop behaviour is 1.4.5's: TryStop is a no-op and the address-scoped PID signal is its stop mechanism.
  • Failed MCP mutating calls are now flagged as tool errors. The adapter decided success by "non-zero exit with stdout ⇒ informational negative" — but every mutating subcommand prints progress to stdout (e.g. " Loading X") before it can fail, so a load/stop/unload that exited 3 with "Error: …" on stderr still came back to the remote agent as a success-shaped result with the error buried in the text. The result mapping is now keyed off the CLI's exit code (TDD §3.3): exit 0 is success, exit 1 stays an informational negative returned as normal content (so status --json's exit-1-with-JSON-array case is unchanged), and exit ≥ 2 — as well as a signal or a failure to run the CLI at all — is a tool error carrying stderr with stdout appended for context. The 1 MiB per-stream output cap 1.4.5 added is unchanged and cannot affect the verdict, which no longer depends on output at all.
  • Two residual false-drift cases are gone from the idempotent-reload notice. 1.4.5 stopped comparing fields llama-server's `/prop...
Read more

v1.4.5

Choose a tag to compare

@airiclenz airiclenz released this 07 Jul 14:25

Security

  • HTTP responses from probed servers are now read with a size cap. The launcher discovers servers by address, so a hostile process squatting on a configured port is an in-scope adversary — and health checks, model listing, and llama-server's /props consumed the response body with no size limit, bounded only by the 2-second client timeout. Since discovery probes several addresses in parallel (on every menu refresh tick, too), one squatter streaming data over loopback could force large concurrent allocations in the one-shot CLI or the long-lived MCP adapter. Every backend body read now goes through io.LimitReader: 8 KiB for status-like payloads (health and backend-discrimination probes, error-message bodies, drained load/unload responses) and 1 MiB for structured payloads that scale with server state (model lists, /props with its embedded chat template). A body past the cap arrives truncated and simply fails the parse that follows, so an oversized health or model-list response is rejected instead of consumed.
  • The MCP adapter's HTTP listener now enforces connection timeouts, and tool output is capped at 1 MiB per stream. The http.Server previously set no timeouts, so a stuck or hostile client could hold connections open indefinitely; it now sets ReadHeaderTimeout (10 s), IdleTimeout (2 min), and WriteTimeout (10 min — generous because load_profile legitimately waits up to 5 minutes for a model load). Separately, the adapter accumulated the CLI's stdout/stderr into unbounded buffers, so a runaway subprocess (e.g. a logs invocation that never stops producing output) could grow the adapter's memory and the MCP response without limit; each captured stream is now capped at 1 MiB, with content past the cap dropped and a [output truncated: 1MiB cap reached] notice appended so the caller knows the output is incomplete.
  • The MCP adapter now validates tool-supplied target/profile arguments before forwarding them to the CLI. tail_log, stop_server, and unload_model passed their optional argument to the CLI as a raw positional, so a hostile (e.g. prompt-injected) client could smuggle a flag or a subcommand keyword through it: tail_log with target: "clean" executed the destructive logs clean through the read surface — available even under --read-only — and target: "-f" made the adapter block forever on logs --follow. All three tools now pass their positional through a single validation gate that rejects values starting with - or matching a launcher subcommand keyword (clean, load, stop, …), returning an MCP tool error without invoking the CLI. Legitimate backend names and host:port targets are forwarded unchanged; the CLI's own logs/stop/unload parsing is untouched (ADR-0008).

Changed

  • Discovery no longer issues an extra GET /props request per llamacpp instance. DiscoverRunningInstances populated a RunningInstance.ResolvedParams field that nothing ever read — at the cost of a live /props query against every reachable llama-server on every discovery pass, including each menu refresh tick. The write-only field and its probe are removed, so discovery now makes one fewer round trip per llamacpp instance; the load-time drift check (ADR-0007) keeps its own /props query, so nothing that a user sees changes. Three unused internal helpers (IsServerAlive, FormatMemoryLine, percentString) were deleted in the same pass.

Fixed

  • A managed llama-server that dies during startup is now detected instead of reported as running. startManagedServer forked the child but never reaped it, so a fast-exiting server (port already in use, bad args, missing model) became a zombie — and a zombie still satisfies kill(pid, 0), so the ~500 ms liveness check reported the dead process as alive and StartServer returned a RunningInstance for a server that had already exited (the real failure only surfaced later as a health-check timeout). The child is now reaped by a cmd.Wait goroutine whose result feeds the grace-period check: if the process exits within the startup window the launcher returns the "server exited immediately after start" error with the log tail, and if it stays up the goroutine stays parked to reap it whenever it does exit, so no zombie is left behind.
  • llama-launcher list now aligns the favourite marker by display width. The list command measured its column widths with byte length (len) while the interactive menu uses visible (rune) width, so a profile description or title containing a multi-byte character — the em dash in the shipped example, for one — pushed the column out of alignment relative to the ASCII rows. The column widths (and the marker's right-alignment padding) are now measured with the same visibleWidth helper the menu uses, so the marker lines up regardless of multi-byte content.
  • Closing a pop-up now restores the terminal cursor. showPopup hid the cursor while displaying a pop-up but never re-showed it, so with the default auto_close: true, opening any pop-up (e.g. "Show model config" or an error) and pressing a key exited the process with the cursor left invisible until the next reset. The popup-exit path now emits the show-cursor escape after the keypress, mirroring the interactive menu's cursor restoration.
  • Auto-stop / auto-unload now identifies the target instance by address and backend. When activating a profile, LoadProfile skips the instance it is about to (re)use so it does not tear that server down — but it matched only on host:port, so a different backend squatting on the profile's address (e.g. a leftover llama-server on :8080 while activating an ollama profile bound there) was mistaken for the target and skipped. With auto_stop_server: true the launcher then stopped every other instance but left the actual blocker running, and the subsequent start failed to bind the port. Both the auto-stop and auto-unload loops now treat an instance as the target only when its address and backend both match; a same-address instance of a different backend is a blocker and is stopped/handled (ADR-0004, ADR-0006).
  • The interactive menu now treats the instance with a model loaded as its "loaded" instance. When several servers were running, the menu picked the first discovered instance as its primary regardless of whether that one had a model — so with an idle LM Studio (which sorts first) alongside an Ollama instance actually serving a model, the menu surfaced the idle server's log and config and rendered an empty Model: line in simple mode. Selection now returns the first discovered instance that actually has a model loaded, falling back to the first instance only when none has one.
  • Stopping an Ollama instance no longer runs a broken CLI call or a host-wide process sweep. Ollama.TryStop shelled out to ollama stop with no model argument — which the ollama CLI rejects, so the hook errored on every call — and then fell back to pgrep -f "ollama serve" and SIGTERMing every matching PID, i.e. every Ollama instance on the host regardless of the target address (that fallback survived only because the errored return was swallowed). TryStop is now a no-op: the launcher already stops the specific instance by signalling whatever process is listening at the target host:port (the address-scoped lsof/PID path in EnsureStopped), which is per-instance and version-independent. It deliberately does not use ollama stop <model> (that only unloads a model from a still-running ollama serve, so it would not free the listener, and the subcommand is absent on older ollama versions). Separately, EnsureStopped now surfaces a backend stop-hook error to stderr instead of discarding it, while still treating the hook as best-effort and non-blocking, matching the documented behaviour (TDD §6.5).
  • status --json now reports every running instance, not one per backend. With auto_stop_server: false two instances of the same backend can run on different ports, and human status lists both — but the JSON path (and therefore the MCP server_status tool) took the first discovered instance per enabled backend and silently dropped the rest. The array now carries one entry per discovered running instance (each with its own address), plus a running: false entry for every enabled backend with no running instance. The per-entry field set and the exit-code semantics (0 if anything is running, else 1) are unchanged.
  • LM Studio profiles now actually apply batch_size and flash_attn, and no longer claim an unsupported gpu_layers mapping. The load call sent only model and context_length, while the config's parameter table and the "Show model config" pop-up claimed gpu_layers (99→"max", 0→"off"), batch_size (→eval_batch_size), and flash_attn were in effect — so e.g. gpu_layers: 0 on an lmstudio profile still loaded with LM Studio's GPU default. batch_size and flash_attn are now forwarded to POST /api/v1/models/load under LM Studio's REST field names (eval_batch_size, flash_attention). gpu_layers could not be implemented: LM Studio's REST load endpoint accepts no GPU-offload field (its only GPU knob, offload_kv_cache_to_gpu, controls KV-cache placement, not layer offload — GPU offload ratios exist only in the lms CLI and SDKs). The parameter table and the pop-up therefore no longer present gpu_layers as lmstudio-applicable; configure GPU offload in LM Studio itself.
  • Configured sampling parameters now actually reach llama-server. The five sampling settings — temperature, repeat_penalty, top_k, top_p, min_p — are documented as llamacpp-applicable and shipped in the example config's defaults:, but the llamacpp argument builder never emitted them, so llama-server always ran with its own built-in sampling defaults and user-configured values were si...
Read more

v1.4.4

Choose a tag to compare

@airiclenz airiclenz released this 13 Jun 14:10

Optional MCP control-plane adapter (llama-launcher-mcp)

A separate, optional host-side binary (make build-mcp) that exposes llama-launcher's lifecycle commands as MCP tools over HTTP, so a client on another machine — typically a coding agent in a container — can control which model runs on the host. It shells out to the CLI and dispatches control commands only, never proxying inference, so llama-launcher itself keeps no listener and ADR-0002 stays intact (see ADR-0008).

  • Tools: list_profiles, server_status, tail_log (read) plus load_profile, unload_model, start_server, stop_server (mutating, omitted under --read-only).
  • Access: source-IP allowlist, no token — --allow <ip|cidr|host> or --allow-interface <name> to allow a local bridge's subnet (covers any IP the bridge assigns the container, nothing to pin). Listener binds the container-facing interface via --listen (not 0.0.0.0).

The Homebrew formula installs the llama-launcher CLI only; build the adapter with make build-mcp.

v1.4.3

Choose a tag to compare

@airiclenz airiclenz released this 11 Jun 13:16

Per-server API keys: servers: entries now accept {enabled: true, api_key: "secret"}. For llama.cpp the key is passed as --api-key so the server requires Authorization: Bearer; for LM Studio it lets the launcher keep working with Require API token enabled. 401/403 responses produce an actionable error, keys are whitespace-trimmed with a warning, and --api-key values in extra_args are masked in the config pop-up.