Skip to content

Releases: llmmanorg/llmman

b317

Choose a tag to compare

@github-actions github-actions released this 04 Sep 21:51
a52ce59

feat(serve): pool hardware across daemons as an aggregation (#405)

Name peer daemons in [aggregation] peers (or LLMMAN_PEERS) and a
request to any node is served by whichever has the model loaded, or the
most room to load it. Forwarded requests carry x-llmman-hop so they are
never forwarded again; ps, tags, models and stop cover every node.

Fixes #404.

b316

Choose a tag to compare

@github-actions github-actions released this 04 Sep 20:53
9e6cc5c

serve: add /api/embed, /api/create, /api/copy, /api/blobs; fix 415, 501 (#403)

Integrating llmman into ~100 Ollama-API clients kept hitting the same
gaps. Each fix follows ollama's server/routes.go and llm/llama_server.go.

Embeddings: /api/embed and /api/embeddings ride on the backend's
/v1/embeddings with ollama's semantics (truncate, dimensions, keep_alive,
bounded fan-out, NaN/Inf rejection). An embedding model is detected by
its GGUF pooling_type key and loaded with --embeddings and a per-slot
batch, which also fixes the 501 /v1/embeddings gave.

Model management: /api/copy is llmman cp over the wire. /api/create
supports from (alias) and files (GGUFs uploaded via /api/blobs, built
like llmman build); Modelfile fields are refused with a 400 naming
them. A loaded model whose tag now points elsewhere is evicted.

Content-Type: the Ollama routes accept a JSON body under any header, as
gin's ShouldBindJSON does, instead of a 415. A cross-site non-JSON POST
from an origin CORS wouldn't allow is refused (it skips preflight).

Also fixes llmman cp/build storing a bare destination verbatim while
run/rm/show resolve one to docker.io/ai/, so cp gemma4 mine && run mine never worked.

b315

Choose a tag to compare

@github-actions github-actions released this 04 Sep 12:52
0e7a3ed

feat(config): add a config subcommand for editing llmman.conf (#401)

Everything llmman.conf holds has to be typed into an editor today, and
the format is deny_unknown_fields, so a misspelled api_kye is a
credential that silently never takes effect and surfaces later as
someone else's 401. Nothing even tells a user where the file is.

Add llmman config, git config verb for verb: list, get, set,
unset and edit, with --system/--user/--file picking the
location. Reads see both files merged, later overriding earlier; writes
go to the per-user file.

Keys are TOML dotted keys, so nothing needs keeping in sync as the
format grows. Every write goes through config::parse before the file
is replaced, turning the typo above into an error next to the thing that
caused it and leaving the file on disk as it was. A file that ends up
holding an api_key is made owner-only, since llmman would otherwise
ignore the key just written; one holding no key keeps its mode, because
/etc/llmman/llmman.conf has to stay readable. Reading a file whose
keys llmman would drop warns rather than reporting a value that will
never be spent.

toml_edit rather than serializing a Conf back out: this is a file
people also hand-edit, and its comments and layout have to survive.

b314

Choose a tag to compare

@github-actions github-actions released this 04 Sep 12:31
beeb37e

docs: add "Why llmman?" section and move metrics reference (#402)

The README opened with what llmman does but never said why someone
would pick it over Ollama, the tool most readers already have. Add a
short section right after the intro that states the four claims that
matter: provider agnostic, registry agnostic, vanilla llama.cpp/GGUF/
safetensors with no private blob format, and one-step transfer from
HuggingFace into your own registry for air-gapped or compliance-bound
setups. A table lays the same points out side by side with Ollama.

The README also carried the full Prometheus metric table, scrape config
and PromQL hints inline under "Serve", which is operator reference
material rather than something a first-time reader needs. Move it to
docs/metrics.md next to configuration.md and verification.md, and leave
a short pointer behind. Content is unchanged apart from headings.

b313

Choose a tag to compare

@github-actions github-actions released this 04 Sep 10:40
c601c71

test: rename has_host tests to describe resolve behavior (#387)

The two tests named after has_host outlived the function: the rule now
lives in is_host_component. Name them after what they assert about
resolve() so the names stay true as the helper moves.

b312

Choose a tag to compare

@github-actions github-actions released this 04 Sep 10:24
5462ceb

Clarify context length backend behavior (#388)

  • Use ASCII apostrophes in context docs

  • Split context length backend docs

b311

Choose a tag to compare

@github-actions github-actions released this 03 Sep 23:46
2e6cd3c

docs: describe launch as exec'ing an agent, not Claude Code (#399)

llmman launches any supported agent, so naming Claude Code in the
walkthrough makes the tool sound Claude-specific. The example command
above already shows Claude Code as one concrete case.

b310

Choose a tag to compare

@github-actions github-actions released this 03 Sep 23:39
ef6ab03

docs: drop marketing claims from README intro (#398)

The intro repeated subscription/vendor-independence claims that are not
what the project needs to communicate up front. Removing them keeps the
opening focused on what llmman actually does.

b309

Choose a tag to compare

@github-actions github-actions released this 03 Sep 23:16
c112e70

feat(config): merge every config file into one llmman.conf (#396)

Provider API keys could only be supplied as environment variables. That
fits a session you started yourself and not a login shell, a systemd
unit, or a machine with a dozen providers configured, where the
alternative is exporting secrets from a shell rc file.

Add a [providers.<id>] table holding api_key, keyed by the id
llmman providers prints rather than by the variable, which is
models.dev's naming and not llmman's. The environment still wins, as it
does for aws and gh: the file is the standing answer and an export is
the deliberate, this-session-only override. Resolution goes through one
choke point, so the CLI and the daemon pick a key up the same way, and
every "no API key" message now names both places to put one.

Rather than add a third config file, fold shortnames.conf and
verify.conf in as [aliases] and [verify]. One file is parsed once
and shared, but a parse failure still means different things to
different readers: verification refuses to run, because a policy llmman
cannot read must not be mistaken for one that does not exist, while
aliases and keys degrade to none. The error is reported once, centrally.

Search only /etc/llmman/ and ~/.config/llmman/, on every platform, with
no environment variable to move them. The /usr/share, install-tree and
beside-the-binary tiers never had anything shipped to them and are
world-readable, which llmman.conf cannot be.

A file carrying a key must not be readable by group or other, the way
ssh refuses a loose private key. The check gates the keys alone, not the
read, so a world-readable /etc/llmman/llmman.conf still supplies the
aliases and trust policy that are not secrets.

b308

Choose a tag to compare

@github-actions github-actions released this 03 Sep 22:43
50e2834

docs: refer to agents rather than coding agents (#397)

llmman's launch integrations point any agent at a local or hosted
model; nothing about the mechanism is specific to coding work. The
"coding agent" phrasing in the README tagline, quick start, launch
module doc and a modelpack comment understated that scope, so drop
the qualifier.