Berth v0.3.4

The gateway can now switch the card and give it back without a prompt (/v1/berth/switch,
/v1/berth/stop), read the whole node in one call (/v1/berth/board), and fall back to an engine
that needs no card while the primary starts. berth pair pairs a phone or browser with the harbor.
Two experimental additions: Linux AMD support, contributed by @jenerestain, and passing the card to
a VM with berth vfio. Windows testers' reports fixed the stowaway list, the CPU meter and engine
rates. One upgrade note: the gateway no longer forwards the node's API key to engines (see Security).

Added

• Gateway: POST /v1/berth/switch and POST /v1/berth/stop switch the card to an engine or give it back without
  sending a prompt. Stop never touches a pinned holder. The /gui/ board's Launch now uses the same switch.
• berth pair pairs a phone or browser with the harbor: a one-time link and its QR code, a
  Pair this device page, and a device cookie for a year. --list, --revoke ID and --revoke-all
  manage devices. berthd keeps only hashes. Pairing is the groundwork for switching engines from the harbor.
• Gateway: GET /v1/berth/board reads the node in one call: the card and who holds it, every engine's state and
  actions, and each engine's models with which are loaded.
• Linux AMD support (experimental), contributed by @jenerestain. berth daemon --gpu-backend amd
  governs a discrete RDNA3 or RDNA4 card, or a Phoenix APU (Radeon 780M), read from sysfs and DRM
  fdinfo with no ROCm. --gpu N names DRM cardN. On an APU, reserved VRAM and GTT (shared host
  RAM) are tracked apart, only whole-card leases are granted, and parking is refused. auto keeps
  NVIDIA whenever nvidia-smi is installed, so existing NVIDIA setups are unchanged.
• Card passed through to a VM (experimental). While the governed card is bound to vfio-pci,
  berth status and its JSON show it as passed through to a VM, and waiters are told why nothing is
  granted. While a VFIO hold is granted, berthd does not open the card, so a driver switch can run.
  card.passthrough and card.returned events mark each change.
• The harbor shows a card passed through to a VM (experimental): a GPU IN VM tag on every tab, the
  card's panel and the departures board marked, the change named in the log, and the session left as a
  gap in the tide.
• berth vfio status and berth vfio run (experimental, Linux only) for passing the governed card to
  a VM. status shows IOMMU, the card's IOMMU group with each function's driver, and the processes
  holding the card. run -- CMD drains the card, stops berthd's own probe for the hold, refuses while
  any process holds it, runs your own switch script with BERTH_VFIO_DEVICES set, and releases the
  hold only when the card is back on its host driver. --before and --after run your own commands
  around the session, for example to stop a service that holds the card and start it again once the
  card is back. Berth never switches drivers or asks for passwords. See docs/vfio.md; contrib/vfio
  has an example launcher and the libvirt device entries.
• Gateway fallback routing: an [[engine]] may name a fallback, another engine in the file that
  needs no card (a CPU llama-server, an Ollama on another box), and fallback_after (10 s by
  default). A request that has waited that long for its own engine is proxied to the fallback when
  the fallback answers its probe at that moment, with the body's model renamed when fallback_model
  is set. A non-streaming answer carries X-Berth-Fallback: <fallback>; a streaming one, whose SSE
  preamble was already sent, gets the fallback's stream after one : berth: fallback <fallback>
  comment. The gateway never starts, leases or drains the fallback, and the primary's start is never
  cancelled, so later requests land on it. The fallback never receives the node's API key, only its
  own key from its entry when it has one. One hop: berth config check refuses chains.
• Cgroup attribution closes the one known stowaway window (chaos scenario 3: a child
  whose parent exits within one 2 s tick). On Linux with a systemd user manager,
  berth run execs the engine in its own transient cgroup scope (systemd-run --user --scope, unit berth-run-<name>-<rand>), which every child inherits on fork: a
  child that escapes its process group and reparents stays the lease's, the lease
  turns stuck naming it, berth reclaim lists it, and a whole-card request is never
  granted contested over it. A GPU process in a scope no live lease recorded is a
  stowaway as before, and a process outside every scope is never attributed to a
  scopeless lease. Without a user manager, inside a Type=notify service (where
  systemd only accepts readiness from the service's own cgroup), and on Windows, the run
  starts as before and the window stands.
• kv_snapshots and kv_snapshots_mib on an [[engine]] entry: the drain saves an idle slot's
  prompt cache into the engine's --slot-save-path directory once the boundary is reached, keeps
  the directory within its budget (16 GiB by default, above the library free floor), and the
  gateway restores the newest snapshot of the engine's model into its empty slots before the first
  request, so a sequence of agents skips the prefill after each switch. The snapshots are keyed by
  the model the engine itself names on /v1/models, so an engine that lists none or many
  (llama-swap) never snapshots, and nothing crosses models. The restore also warms the files into
  the page cache during the start's wait: cached, never reserved. A save is bounded at 60 s and a
  restore at 30 s, so whatever goes wrong along the way costs one line and the switch goes on as
  before. A snapshot holds the conversation's tokens: the ledger is owner-only, and the README asks
  for a private directory.
• park_ttl on an [[engine]] entry sets how long that engine stays parked before Berth stops it,
  in place of [park] ttl. With keep_comfyui_up = true and a short park_ttl, a ComfyUI node that
  calls another engine (a chat panel calling llama-swap) gets its answer. The drain leaves ComfyUI
  up, and Berth stops the server once the ttl passes
  (#121).

Changed

• The chat shelf is now called text: the shelf names the kind of engine that loads a model, and
  embedders, rerankers and vision or OCR models sit there too. berth models, its JSON and the
  harbor say text. shelf = "chat" in berth.toml, --shelf chat and catalogs from older
  machines still read as text. Asked by @azrael-229
  (#148).
• The README keeps the overview, quick start and screenshots, and the reference moved to pages under
  docs/: install, engines, leasing, models, gateway, the harbor, interfaces, benchmarks, roadmap and
  FAQ. The README's old Tested hardware, Swap numbers and Roadmap headings link to their new pages.

Security

• The gateway no longer forwards the node's API key to engines. A proxied request, fallback and
  ComfyUI routes included, carries the engine's own key when its entry has one (api_key_env or
  api_key_file) and no Authorization when it has none. If an engine of yours checked the node's
  key, give it its own key in its berth.toml entry.

Fixed

• Windows: the host's CPU is read (GetSystemTimes), so the harbor's machine meter and berth status show real CPU
  use instead of 0%. Windows has no load averages: the node now leaves them out of its JSON and the harbor and
  berth status say n/a instead of three zeros. Reported by @azrael-229 (#139).
• One empty models--* husk no longer makes a whole store read as an HF cache with no models,
  reported by @azrael-229 (#138). A root is an HF cache only when a models--* directory holds
  snapshots, so an abandoned download beside plain models keeps them listed.
• The gateway's /gui/ board launches an engine that has a start command but no gui URL (Strata, a
  llama-server). Its Launch button answered 404 and stayed on "Launching..."; it now starts the engine under a
  lease and shows Running once it answers. A refused start shows its reason instead of hanging.
• berth vfio status ADDR asks berthd for its view instead of calling it unreachable, and while the
  card is in a VM it says so instead of printing the last memory reading as current.
• The wake page and start errors show an engine's last log line without terminal color codes
  ([32m[INFO][0m), and a progress bar's line as its last redraw.
• Windows: a process the GPU counters report but nvidia-smi does not list, such as dwm.exe
  holding the card's memory, now shows its name instead of a blank one.
• The harbor engine cards and berth status draw an unreported rate as unknown, not zero. An
  engine whose counters endpoint never answered (a llama-swap whose llama-server starts without
  --metrics) shows n/a for decode, prefill and the token totals, and the ENGINE table prints a
  dash. Reported by @azrael-229 (#140).
• Windows: desktop apps no longer fill the stowaway list. nvidia-smi lists every graphics client with
  no memory figure, so explorer, Chrome, OneDrive and 30 more showed as 0.0 GiB stowaways. A listed
  process now needs the same 64 MiB in the GPU counters as an unlisted one. dwm.exe, whose counter
  reports a committed reservation of 11 to 14 GiB, counts as the desktop baseline, as Xorg does on
  Linux, so DOCKED and STOWAWAY no longer add up past the card. Reported by @azrael-229
  (#149).
• An engine card no longer reads 0.0 t/s for most of a llama.cpp generation. llama.cpp adds a
  request's tokens to its counters only when the request ends, so the decode and prefill rates
  sat at 0 and spiked once per request. While a request runs, the card now shows the rate the
  last finished request measured; an idle engine still reads 0. Reported by @azrael-229
  (#147).