Skip to content

gpu-broker 0.4.0

Choose a tag to compare

@aphilips aphilips released this 03 Oct 18:06
783e01c

Added

  • gpu-broker setup: one command from install to a running broker, asking no questions.
    • Finds the GPU, and llama.cpp, vLLM, Ollama and ComfyUI on their usual ports, with the
      systemd units or Docker containers that run them.
    • Writes the config and catalog for them (the starter files if it finds nothing) and a new
      API token in broker.env (mode 600). Existing files and the token are kept.
    • Installs and starts the gpu-broker service, never as root: a user service (lingering)
      for systemctl --user model servers; else, with root or passwordless sudo, a system
      service run as the installation's owner or a dedicated gpu-broker account, with a
      visudo-checked sudoers rule for exactly the catalog's units. It refuses code anyone else
      could change. Otherwise it prints the serve command, and runs it in the foreground at a
      terminal.
    • Runs check, waits for /health, and opens the dashboard (not over SSH).
    • --dry-run, --yes, --dir. See docs/setup.md.
  • Catalog: health_path for LLM servers without /health (Ollama: /api/version).
  • README: a TLDR block first, and the logo (light and dark).
  • gpu-broker init: writes a starter config.yaml and catalog.yaml to /etc/gpu-broker
    (or --dir) and creates the folders they name.
    • Existing files are kept unless --force.
  • Docs pages: docs/catalog.md, docs/exec-recipes.md, docs/api.md, docs/hardware.md,
    docs/threat-model.md, and AGENTS.md for automated reviewers.

Changed

  • README: leads with the one-person case; installs from PyPI; reference material moved to docs/.
  • Internal: modules that mixed two responsibilities are split, with no behaviour change.
    • catalogschema (entry types and validation) out of catalog.
    • settingsschema (the dataclasses) out of settings.
    • admission (job submit) out of broker.
    • drivers.systemd and drivers.docker out of drivers.local.
    • demo.assemble out of demo.run.
    • The old modules re-export the moved names, except the two concrete drivers: import
      SystemdDriver and DockerDriver from gpu_broker.drivers.systemd and gpu_broker.drivers.docker.