v0.5.0

Added

• Built-in docs. vllm-helper serve now has a Docs tab with a quickstart, the profile and memory knobs, the Coder integration, and a CLI reference generated from the command tree. The pages come from docs/, which stays readable on GitHub.
• Public docs site. cmd/vllm-helper-docs serves the same pages without the panel. It also serves each page's Markdown source, /llms.txt, /llms-full.txt and a sitemap. Dockerfile.docs deploys it on Railway. It is a separate binary, so vllm-helper itself gains no unauthenticated routes.
• Recipes: launch configurations that were run and measured, not inferred. Each one pins the model, the image (by digest), the exact argv, and the numbers it produced on named hardware.
  • recipes list and recipes show say where every setting came from.
  • recipes launch starts containers over SSH on the nodes you name; a recipe can span machines. It writes a profile of the same name and points Coder at what is serving, so the picker and the server always agree. Then it watches the load, like switch does.
  • recipes stop removes the containers.
  • recipes install-service installs a unit that keeps the active recipe serving, plus a timer that relaunches it when the endpoint stops answering.
    • It stands down the native unit, which can't serve a container.
    • It waits while a launch is in flight, so a loading model isn't mistaken for an outage and replaced with the previous profile.
    • A deliberate stop or systemctl stop stays stopped.
  • The web panel shows a recipe's health and log, but its buttons don't drive recipes yet; use the CLI.
• Runner-aware commands. status, logs, restart and stop act on whatever is actually serving. They used to assume the init-system service. That made them report systemd as healthy while a container served, or print "Stopped vllm.service" for a two-node cluster.
• Clean switches. Switching between a recipe and a native profile stops the one being left, so it frees the GPUs and port.
• Declared deviations. A recipe records each difference from its upstream recipe as data: the setting, upstream's value, ours, and why. tools/recipediff reports any undeclared difference. It compares content, not commit ids, so a pin that only trails upstream isn't reported as drift.
• More than one machine.
  • node add registers a machine once, by the URL it serves on and its SSH target. pool, shard and recipes launch then refer to it by name.
  • pool serve puts a router in front of replicas of one model and keeps each conversation on the node that already has its prefix cached. pool install-service runs the router as a unit.
  • shard start splits one model across machines with Ray or vLLM's multiprocessing executor, for trying a split before a recipe exists. shard install-ray prepares the nodes.
  • spark doctor and spark install-prereqs handle the build packages a DGX Spark needs.

Changed

• Coder entries are matched by model, not profile name. A renamed profile, or a second machine serving the same model, now adopts the existing entry instead of adding a duplicate row to the picker.
• doctor warns when the Coder token is within two days of expiring. An expired token silently stops picker updates.
• New setting CODER_MAX_OUTPUT_TOKENS. It raises Coder's server-side reply cap for this machine's models. If it's unset, nothing is sent and Coder's own setting stays.