Repository navigation
Stable launch, tagged 2026-10-12. Multihull deploys one container to Kubernetes and Modal from one multihull.yaml, serves both behind one URL through a Rust router, and fails over between them per request. 0.2.0 adds the demos, hull top, multi-arch images, failover on provider edge errors and a quickstart that a fresh user can follow end to end.
Scope, plainly: a Kubernetes plus Modal failover router for a single model, with Docker as the free local layer. Not yet the five-provider framework the architecture plan describes. 0.1.0 notes: docs/releases/0.1.0.md.
Install
uv tool install 'multihull[modal]==0.2.0'
cargo install multihull --version 0.2.0
helm pull oci://ghcr.io/mishraprafful/charts/multihull --version 0.2.0Images: ghcr.io/mishraprafful/multihull-router:0.2.0, multihull-controller:0.2.0, multihull-mock-server:0.2.0, each a linux/amd64 and linux/arm64 manifest list. The GitHub release carries router binaries for linux amd64, linux arm64 and macOS arm64. Images, chart and packages are public; see Install from OCI.
Highlights
make demo: the five-beat failover demo on three local docker targets, free, no GPUs, no credentials; recorded in the README.make demo-cloudruns the same beats on kind plus Modal for cents.hull top: a live terminal view of the router, per endpoint and per route, from the admin listener.hull --version, and a quickstart rewritten from a fresh-user walkthrough: seven steps through the first request, with the measured time.- Images for
linux/amd64andlinux/arm64under every tag, so an Apple Silicon kind node or Docker target pulls them directly. - The Kubernetes namespace is
multihull:hull initwrites it, the Helm chart creates it,hull deploystops with one line when it is missing. - Provider edge errors fail over: a stopped Modal app answers
404 modal-http: invalid function callfrom the edge for about 14 s before the probe ejects it; the router now retries those on another provider, even for a POST without anIdempotency-Key. A model's own 404 is still returned. - Docs version switcher, Artifact Hub metadata on the chart, gitleaks on every PR, base images from Docker Hub mirrors pinned by digest.
What is proven
By live runs:
- Cloud demo on 2026-10-10 (
make demo-cloud, kind primary, Modal secondary, mock server on CPU, 180 s scripted): 511 requests, 0 client errors, 139 failovers. Deploy to both Ready in 14 s with the Modal image cached. The primary scaled to zero at 40 s, the circuit opened 12 s later and the probe marked it down 5 s after that with every request in between served by Modal; the restore at 70 s reachedhalf_openat 94 s andclosedat 99 s, then traffic returned to kind. Modal bill 0.0012 USD. Recorded indocs/runbooks/demo-cloud.md. - A real model on GPUs: two consecutive live GPU runs on
mainon 2026-10-08 (37821488815, 37822124427,live-gpu.yml) deployedQwen/Qwen2.5-1.5B-Instructon vLLM 0.11.0 to two Modal L4 targets, streamed tokens over SSE and plain completions, stopped the primary app and served every failover request from the secondary with zero client errors (23 requests per run). Deploy reached Ready in about 100 s with the image cached. These runs predate the edge-404 fix; the failover test now asserts zero 404s aftermodal app stopand waits for an owner run. - Fresh-user walkthrough of the quickstart on macOS arm64 against a kind cluster: six minutes of wall clock, cluster creation and a router build included;
make demofrom a clean clone in 73 s (182 requests, 0 errors, 91 failovers). The chart was installed from the published OCI package.
On every PR and push to main:
- The local e2e suite, 36 rows in about 4 minutes; the kind suite, 51 rows (kind primary, docker secondary); the in-cluster suite, 9 rows (controller and router from the chart in kind over mTLS, controller restarts with kept and wiped state); the scripted
make demo, 60 s with a kill and restore and zero client errors. - Unit and golden tests for the control plane (361 tests) and the router.
Not proven:
- Kubernetes with a GPU: kind has no GPUs, so node selectors and tolerations have never scheduled on a real cluster, and no real model has run on Kubernetes.
- A cluster pull from a private registry.
- RunPod, Baseten and Replicate are render-only:
hull planworks,applyraises not implemented. - Only SQLite state exists.
s3://,gs://andpostgres://exit with a not-implemented error. - No sustained load or multi-replica router measurements. The plan's latency and throughput numbers are targets, not results.
- The chart ships no RBAC for the controller.
- No outside user has run
hull initthroughhull deploy; the walkthrough above was done by a maintainer.
Most of this code was written by coding agents and reviewed by CI and other agents. A human read-through of the router's retry and circuit paths (router/crates/multihull/src/proxy/handler.rs, core/circuit.rs, core/retry.rs) is recommended before relying on it in production.
Upgrade notes
From 0.1.0:
- Namespace.
hull initnow writesnamespace: multihull; a spec written by 0.1.0 keeps whatever it has (inferencefrom the oldhull init, ordefaultwhen unset) and keeps deploying there.hull deployfails before applying when the target namespace does not exist; create it, or install the chart, which createsmultihull. - Chart
workloads.*values. Chart 0.2.0 renders amultihullNamespace (workloads.namespace,workloads.createNamespace: true, labelmultihull.dev/managed-by: multihull,helm.sh/resource-policy: keep) unless it equals the release namespace. If amultihullNamespace already exists outside Helm, the upgrade fails to adopt it: setworkloads.createNamespace=falseorworkloads.namespaceto another name.helm uninstallleaves the Namespace and the workloads in it. edge_errorsnapshot field.Endpoint.edge_erroris optional indiscovery.protoand the snapshot JSON. A 0.2.0 router without it behaves as 0.1.0 did; a 0.1.0 router ignores it. Mixed versions of controller and router work either way, and the Modal translator sets it on every new snapshot.- Images. The
0.2.0tags are manifest lists forlinux/amd64andlinux/arm64. Local builds andkind loadworkarounds for Apple Silicon are no longer needed;charts/multihull/Chart.yamllists both platforms underartifacthub.io/images. hull deployis a dry run without--apply, as it was in 0.1.0; the docs now say so.
Pull requests
- docs: record the 0.1.0 release by @mishraprafful in #125
- feat(chart): prepare the Helm chart for an Artifact Hub listing by @mishraprafful in #126
- ci(secrets): scan every PR and push to main with gitleaks by @mishraprafful in #129
- feat(website): add a docs version switcher by @mishraprafful in #128
- docs(handover): keep session entries in one file per session by @mishraprafful in #133
- fix: resolve the CodeQL alerts on main by @mishraprafful in #132
- feat(cli): add hull top, a live view of router endpoints and routes by @mishraprafful in #131
- feat(demo): add make demo, the local failover demo by @mishraprafful in #130
- docs: drop the GHCR token steps now that the packages are public by @mishraprafful in #135
- docs(handover): keep the handover short and turn the session log into lessons by @mishraprafful in #134
- feat(demo): add make demo-cloud, the kind and Modal failover demo by @mishraprafful in #136
- feat(demo): record the local demo with vhs and embed it in README and docs by @mishraprafful in #137
- docs(security): state the supported versions now that 0.1.0 is released by @mishraprafful in #144
- docs: launch checklist, badges, code of conduct and milestone names by @mishraprafful in #143
- ci: pull base images from Docker Hub mirrors pinned by digest by @mishraprafful in #142
- feat(cli): add --version by @mishraprafful in #138
- fix(router): fail over on provider edge 404s instead of returning them by @mishraprafful in #146
- chore(launch): add the social preview image and record a cloud demo run by @mishraprafful in #153
- test(e2e): wait for scale attempts on every target before comparing them by @mishraprafful in #151
- fix(kubernetes): standardise on the multihull namespace owned by the chart by @mishraprafful in #154
- docs(quickstart): fix stumbles from a fresh-user walkthrough of main by @mishraprafful in #155
- ci(images): publish linux/amd64 and linux/arm64 manifests by @mishraprafful in #156
- chore(release): prepare 0.2.0 by @mishraprafful in #157
Full Changelog: v0.1.0...v0.2.0
- docs: record the 0.1.0 release by @mishraprafful in #125
- feat(chart): prepare the Helm chart for an Artifact Hub listing by @mishraprafful in #126
- ci(secrets): scan every PR and push to main with gitleaks by @mishraprafful in #129
- feat(website): add a docs version switcher by @mishraprafful in #128
- docs(handover): keep session entries in one file per session by @mishraprafful in #133
- fix: resolve the CodeQL alerts on main by @mishraprafful in #132
- feat(cli): add hull top, a live view of router endpoints and routes by @mishraprafful in #131
- feat(demo): add make demo, the local failover demo by @mishraprafful in #130
- docs: drop the GHCR token steps now that the packages are public by @mishraprafful in #135
- docs(handover): keep the handover short and turn the session log into lessons by @mishraprafful in #134
- feat(demo): add make demo-cloud, the kind and Modal failover demo by @mishraprafful in #136
- feat(demo): record the local demo with vhs and embed it in README and docs by @mishraprafful in #137
- docs(security): state the supported versions now that 0.1.0 is released by @mishraprafful in #144
- docs: launch checklist, badges, code of conduct and milestone names by @mishraprafful in #143
- ci: pull base images from Docker Hub mirrors pinned by digest by @mishraprafful in #142
- feat(cli): add --version by @mishraprafful in #138
- fix(router): fail over on provider edge 404s instead of returning them by @mishraprafful in #146
- chore(launch): add the social preview image and record a cloud demo run by @mishraprafful in #153
- test(e2e): wait for scale attempts on every target before comparing them by @mishraprafful in #151
- fix(kubernetes): standardise on the multihull namespace owned by the chart by @mishraprafful in #154
- docs(quickstart): fix stumbles from a fresh-user walkthrough of main by @mishraprafful in #155
- ci(images): publish linux/amd64 and linux/arm64 manifests by @mishraprafful in #156
- chore(release): prepare 0.2.0 by @mishraprafful in #157
Full Changelog: v0.1.0...v0.2.0