v0.6.0

Added

• Providers: a profile can run on an inference server other than vLLM. The
  first is MLX (mlx_lm.server from mlx-lm) on Apple Silicon. A profile
  records its server in PROVIDER=, and one without it is vLLM, so every
  existing profile serves as before. PROVIDER in config.env picks the
  server for new profiles only, and setup --provider mlx writes it with
  MLX_LM_BIN. vLLM-only steps (model-card flags, parser checks, KV sizing,
  GGUF quants, shards) are skipped or refused for an MLX profile. See
  docs/providers.md.
• llama.cpp provider: llama-server serves GGUF models on Metal, CUDA or the
  CPU. setup --provider llamacpp writes LLAMA_SERVER_BIN, and
  new <repo> --quant Q4_K_M picks one GGUF file out of a repo. Unlike MLX it
  checks VLLM_API_KEY, so it listens on every interface as vLLM does; with
  no key set it stays on 127.0.0.1. The key reaches llama-server as
  LLAMA_API_KEY in its environment, not as --api-key, so ps does not
  show it.
• macOS serves on the Apple Silicon GPU through vllm-metal. llm-helper asks
  the Python beside VLLM_BIN whether vllm_metal is importable and reads the
  Metal working set from MLX; setup and doctor say which backend is in use.
  Under vllm-metal, new profiles default GPU_MEMORY_UTILIZATION to 0.9 of
  the working set, the model budget is measured against the working set rather
  than RAM, and doctor skips the source-build toolchain. setup finds the
  vllm that Homebrew and the install script put off PATH. A headless Mac
  runs the server as a LaunchDaemon with SERVICE_SCOPE=system. The CPU
  backend still works where vllm-metal is not installed. vllm-metal 0.32.0
  ignores --kv-cache-memory-bytes, so there new leaves the KV cache to
  vllm-metal and says so, and plans a size only for builds that honor it.
• metal-limit shows and sets the GPU's wired memory limit on macOS, and
  installs a boot-time daemon to reapply it, since the sysctl resets on
  reboot. A higher limit gives vllm-metal more of the RAM at the cost of
  headroom for macOS.
• node add describes the node's hardware over SSH (memory and NVIDIA GPUs,
  using only /proc/meminfo and nvidia-smi there, so llm-helper need not
  be installed on it), and node ls shows it. A node that cannot be reached
  is still added; node probe describes it later, and --no-probe skips it.
• Hardware definitions: what llm-helper knows about specific machines (how a
  GPU's memory is shared, its architecture, the flags that abort there, PCI
  vendors) is data in catalogue/hardware/, one file per entry, each with a
  reason, in machines/, architectures/ and vendors/. Files in the same
  folders under hardware.d in the config directory add definitions without
  a rebuild but cannot change shipped ones. hardware list shows the
  definitions in effect and hardware report prints a machine's readings,
  without host identifiers, to contribute a new one. Every shipped machine is
  tested against a report in catalogue/hardware/reports/, and no two may
  match the same GPU. See docs/hardware.md.
• Detection reads more than GPU names: compute capability and addressing
  mode from nvidia-smi, Jetson markers, the PCI bus and the cgroup memory
  limit. doctor shows which definition matched and what decided whether
  memory is shared, lists AMD GPUs and other accelerators it does not drive
  yet, and warns when an NVIDIA GPU has no working driver, when the driver's
  device node is present but nvidia-smi fails, when the machine is sized for
  the CPU backend despite an accelerator, when a hardware.d file
  was skipped, when more than one definition matches a GPU, and when memory
  on a unified machine is already in use.
• Engines: each inference server (vLLM, MLX, llama.cpp) is a definition in
  its own file in catalogue/engines/, named after it, rather than Go code:
  its command, how it takes the model and the API key, where it listens, its
  health check and the reason for each. *.json files in engines.d in the
  config directory add engines without a rebuild but cannot change shipped
  ones. engines list and engines show print the engines in effect. See
  "Contributing an engine" in docs/providers.md.
• Recipes can run engines other than vLLM. A recipe names its engine in
  provider, the hardware it needs in requires, Hugging Face models pinned
  to a revision in models, and engine settings in params. Each shipped
  recipe is its own file in catalogue/recipes/, named after it.
  recipes launch checks each node's GPU vendor against the engine, and
  checks the pinned revisions are fully downloaded in each node's Hugging
  Face cache, printing the hf download command for any that is missing or
  incomplete. It waits on the engine's own health path, asked on the head
  node itself, so an engine that listens only on loopback is still seen to
  serve. *.json files in recipes.d add local recipes, which
  recipes list marks as local. See
  "Contributing a source" in docs/recipes.md.
• TensorFold (tensorfold-python) engine and the
  glm-5.3-flash-tf-2spark recipe: GLM-5.3 Flash sharded across two DGX
  Sparks at a 1M-token context, derived from MiaAI-Lab's launcher. Its
  drafter, incoai/GLM-5.3-Flash-DFlash2, is CC BY-NC-ND 4.0, for
  non-commercial use only.

Changed

• The tool is now called llm-helper. Releases ship the binary as both
  llm-helper and vllm-helper, archives are named llm-helper_*, and the
  help, version, docs and the commands that messages suggest use the new name.
  Nothing on disk changes: the config and state directories, VLLM_HELPER_*
  overrides, VLLM_BIN, VLLM_API_KEY and the vllm.service /
  com.coder.vllm units keep their names, and vllm-helper upgrade from
  0.5.0 and earlier finds the new build as before. That upgrade installs only
  vllm-helper, so link llm-helper beside it as docs/install.md shows.
• spark doctor and spark install-prereqs use the same build toolchain
  checks as doctor. Python headers are looked for where the vllm venv's own
  interpreter expects them, rather than at /usr/include/python3.12, and
  only missing packages are installed.
• recipes list shows the hardware each recipe was measured on instead of
  calling it "this hardware", and a recipe launched on a node that matched
  different hardware says so.
• The docs sidebar, on the docs site and in the panel, folds the CLI
  reference. It opens when the page you are on is inside it, and the arrow
  beside it opens and closes it.

Fixed

• On a machine with more than one NVIDIA GPU only the first was read, so
  doctor, setup and the panel described one card. Every GPU is now listed,
  and the budget, still one GPU's worth because that is what a server without
  --tensor-parallel-size uses, says so.
• CUDA_VISIBLE_DEVICES was ignored when sizing models and fitting
  --gpu-memory-utilization, so a server limited to one card was sized
  against GPU 0. The value the server is started with, from config.env, the
  profile or the environment, now decides which GPU is described, and doctor
  says how many it hides. Servers are started with
  CUDA_DEVICE_ORDER=PCI_BUS_ID, so an index names the GPU nvidia-smi gives
  that index. A shard reads the profile it serves rather than the active one,
  and a switch made from the CLI reaches a running panel.
• Flags known to abort vLLM on an architecture, such as SM100-only kernels on
  a GB10, were only refused when a tested recipe launched. They are now
  refused for start, a profile's EXTRA_ARGS and shards, and a recipe or
  shard checks each node's own architecture. A shard's head is checked against
  the GPU it will run on before any node is stopped.
• With CUDA_VISIBLE_DEVICES hiding every GPU, the budget was still a GPU's
  worth. It is now zero, on a Jetson too, and doctor says why. inspect and
  the panel say nothing fits, rather than that no budget is configured.
• A hung nvidia-smi hung doctor, search and the panel. It now gets 10
  seconds.
• Inside a container or a unit with a cgroup memory limit, the budget was
  sized for the host's RAM. It is now held to the limit.
• Adding a node again to correct its --host dropped its SSH target and
  probed hardware unless --ssh was repeated.
• macOS system-scope LaunchDaemons ran as root, so the server read root's
  config and model cache rather than the ones setup wrote. They now run as
  the installing user, with HOME set. setup keeps an existing plist, so a
  daemon installed by 0.5.0 or earlier has to be removed and set up again;
  docs/macos.md has the steps.
• adopt records the scope of the unit it finds even when the name already
  matches SERVICE_NAME, so a system-scope job under the default name is no
  longer controlled as a user-scope one.
• Running setup again fills in a VLLM_BIN the first run left empty, once
  a vllm can be found, instead of keeping the empty value. A config whose
  PROVIDER names another server is left alone.
• The Python beside a symlinked vllm, as Homebrew installs it, is found by
  following the link. A Homebrew Cellar path is recorded as the stable opt
  path, so brew upgrade no longer leaves VLLM_BIN pointing at nothing.
• A user-scope service that fails to load over SSH now says why: a
  LaunchAgent needs a GUI login session, and launchd reports only error 125.
  The message points at SERVICE_SCOPE=system.
• status and doctor read a macOS system-scope job's state without sudo.
  Going through sudo failed without a terminal, so a serving job was
  reported as inactive. Only start, stop, restart and switch need it.