v0.8.0

Native install (Apple Silicon / homelab)

uv tool install mship
mship bootstrap --metal
mship start --config models.yaml

Docker images

Thin (control/coordinator — default, no torch/vllm)

docker pull ghcr.io/modelship-ai/modelship:0.8.0

CUDA (GPU node)

docker pull ghcr.io/modelship-ai/modelship:0.8.0-cuda

CPU (CPU node)

docker pull ghcr.io/modelship-ai/modelship:0.8.0-cpu

Floating tags (:latest, :latest-cuda, :latest-cpu) are single-node only — for any
multi-node cluster, pin every node to the same X.Y.Z tag to avoid a Ray version
mismatch between head and workers.

Helm chart (Kubernetes)

helm install modelship \
  oci://ghcr.io/modelship-ai/charts/modelship \
  --version 0.8.0 \
  -f values.yaml

Running an image

The images take the same subcommands as the mship CLI:

docker run --rm --shm-size=8g \
  -v ./models.yaml:/models.yaml \
  -v ./models-cache:/.cache \
  -p 8000:8000 \
  ghcr.io/modelship-ai/modelship:0.8.0-cpu start --config /models.yaml

Configuration

Create a models.yaml file, mount it and pass it with --config. Example configs for various GPU sizes are included in the image under /modelship/config/examples/ — use them as a reference for structure and available options, then tailor the models and GPU fractions to your hardware.

What's Changed

• Bench/llama server prompt cache off by @alez007 in #206
• fix: keep vLLM/Triton/FlashInfer caches node-local under MSHIP_NODE_C… by @alez007 in #208
• fix: resolve whispercpp built-in names through the HF cache by @alez007 in #209
• fix: stop forwarding the driver's HF settings to replicas by @alez007 in #210
• Fix/container any uid by @alez007 in #211
• Fix/sherpa onnx concurrent fetch by @alez007 in #212
• docs: add Discord community links across README, docs, and package me… by @alez007 in #213
• refactor: route sherpa_onnx bundles through a shared model-sources pa… by @alez007 in #214
• Feat/download leases by @alez007 in #215
• fix: configure logging in vLLM's engine and worker processes by @alez007 in #216
• fix: pin the cluster-wide actors to the head node by @alez007 in #217
• test: raise max_tokens on the llama_server tool-call tests to fit the… by @alez007 in #218
• Feat/forward cluster settings by @alez007 in #219
• Feat/start join verbs by @alez007 in #220
• Fix/chart kuberay 171 by @alez007 in #221
• Feat/deploy handoff by @alez007 in #222
• fix: a rollback deletes against the version the deploy coordinator holds by @alez007 in #223
• Fix/deploy commit timeout by @alez007 in #224
• refactor: only mship start creates the deploy coordinator by @alez007 in #225
• fix: llama_server rejects a source that isn't a GGUF file at driver p… by @alez007 in #226
• Fix/stop handler ignores repeat signals by @alez007 in #227
• Feat/deploy wait flag by @alez007 in #228
• Fix/stop cancel rollback wait by @alez007 in #229
• Fix/deploy head state store by @alez007 in #230
• fix: logging and metrics are set once on mship start for the whole cl… by @alez007 in #231
• Feat/gateway autoscaling by @alez007 in #232
• Feat/remote deploy by @alez007 in #233
• Fix/k8s gateway readiness by @alez007 in #234
• fix: accept every reasoning_effort value OpenAI defines by @alez007 in #237
• fix: pin the disconnect and heartbeat registries to the head by @alez007 in #236
• fix: start refuses a bad state store or gateway sizing without a trac… by @alez007 in #235
• Fix/release docs by @alez007 in #238
• Fix/kokoro multi lang pin by @alez007 in #239
• fix: a model with replicas or autoscaling must set its context length by @alez007 in #240
• Fix/llama cpu ram budget by @alez007 in #241
• Fix/replica count change replaces by @alez007 in #242
• Fix/vllm structured output reasoning by @alez007 in #243
• Fix/vllm tool parser detection by @alez007 in #244
• Docs/diffusers example sd turbo by @alez007 in #245
• Fix/start config reconciles by @alez007 in #246
• test: seed the two llama_server tool_choice fallback requests by @alez007 in #247
• Fix/hide gpus from zero gpu workers by @alez007 in #248

Full Changelog: v0.7.15...v0.8.0