Repository navigation
Releases
v0.2.0
Stable launch, tagged 2026-10-12. Multihull deploys one container to Kubernetes and Modal from one multihull.yaml, serves both behind one URL through a Rust router, and fails over between them per request. 0.2.0 adds the demos, hull top, multi-arch images, failover on provider edge errors and a quickstart that a fresh user can follow end to end.
Scope, plainly: a Kubernetes plus Modal failover router for a single model, with Docker as the free local layer. Not yet the five-provider framework the architecture plan describes. 0.1.0 notes: docs/releases/0.1.0.md.
Install
uv tool install 'multihull[modal]==0.2.0'
cargo install multihull --version 0.2.0
helm pull oci://ghcr.io/mishraprafful/charts/multihull --version 0.2.0Images: ghcr.io/mishraprafful/multihull-router:0.2.0, multihull-controller:0.2.0, multihull-mock-server:0.2.0, each a linux/amd64 and linux/arm64 manifest list. The GitHub release carries router binaries for linux amd64, linux arm64 and macOS arm64. Images, chart and packages are public; see Install from OCI.
Highlights
make demo: the five-beat failover demo on three local docker targets, free, no GPUs, no credentials; recorded in the README.make demo-cloudruns the same beats on kind plus Modal for cents.hull top: a live terminal view of the router, per endpoint and per route, from the admin listener.hull --version, and a quickstart rewritten from a fresh-user walkthrough: seven steps through the first request, with the measured time.- Images for
linux/amd64andlinux/arm64under every tag, so an Apple Silicon kind node or Docker target pulls them directly. - The Kubernetes namespace is
multihull:hull initwrites it, the Helm chart creates it,hull deploystops with one line when it is missing. - Provider edge errors fail over: a stopped Modal app answers
404 modal-http: invalid function callfrom the edge for about 14 s before the probe ejects it; the router now retries those on another provider, even for a POST without anIdempotency-Key. A model's own 404 is still returned. - Docs version switcher, Artifact Hub metadata on the chart, gitleaks on every PR, base images from Docker Hub mirrors pinned by digest.
What is proven
By live runs:
- Cloud demo on 2026-10-10 (
make demo-cloud, kind primary, Modal secondary, mock server on CPU, 180 s scripted): 511 requests, 0 client errors, 139 failovers. Deploy to both Ready in 14 s with the Modal image cached. The primary scaled to zero at 40 s, the circuit opened 12 s later and the probe marked it down 5 s after that with every request in between served by Modal; the restore at 70 s reachedhalf_openat 94 s andclosedat 99 s, then traffic returned to kind. Modal bill 0.0012 USD. Recorded indocs/runbooks/demo-cloud.md. - A real model on GPUs: two consecutive live GPU runs on
mainon 2026-10-08 (37821488815, 37822124427,live-gpu.yml) deployedQwen/Qwen2.5-1.5B-Instructon vLLM 0.11.0 to two Modal L4 targets, streamed tokens over SSE and plain completions, stopped the primary app and served every failover request from the secondary with zero client errors (23 requests per run). Deploy reached Ready in about 100 s with the image cached. These runs predate the edge-404 fix; the failover test now asserts zero 404s aftermodal app stopand waits for an owner run. - Fresh-user walkthrough of the quickstart on macOS arm64 against a kind cluster: six minutes of wall clock, cluster creation and a router build included;
make demofrom a clean clone in 73 s (182 requests, 0 errors, 91 failovers). The chart was installed from the published OCI package.
On every PR and push to main:
- The local e2e suite, 36 rows in about 4 minutes; the kind suite, 51 rows (kind primary, docker secondary); the in-cluster suite, 9 rows (controller and router from the chart in kind over mTLS, controller restarts with kept and wiped state); the scripted
make demo, 60 s with a kill and restore and zero client errors. - Unit and golden tests for the control plane (361 tests) and the router.
Not proven:
- Kubernetes with a GPU: kind has no GPUs, so node selectors and tolerations have never scheduled on a real cluster, and no real model has run on Kubernetes.
- A cluster pull from a private registry.
- RunPod, Baseten and Replicate are render-only:
hull planworks,applyraises not implemented. - Only SQLite state exists.
s3://,gs://andpostgres://exit with a not-implemented error. - No sustained load or multi-replica router measurements. The plan's latency and throughput numbers are targets, not results.
- The chart ships no RBAC for the controller.
- No outside user has run
hull initthroughhull deploy; the walkthrough above was done by a maintainer.
Most of this code was written by coding agents and reviewed by CI and other agents. A human read-through of the router's retry and circuit paths (router/crates/multihull/src/proxy/handler.rs, core/circuit.rs, core/retry.rs) is recommended before relying on it in production.
Upgrade notes
From 0.1.0:
- Namespace.
hull initnow writesnamespace: multihull; a spec written by 0.1.0 keeps whatever it has (inferencefrom the oldhull init, ordefaultwhen unset) and keeps deploying there.hull deployfails before applying when the target namespace does not exist; create it, or install the chart, which createsmultihull. - Chart
workloads.*values. Chart 0.2.0 renders amultihullNamespace (workloads.namespace,workloads.createNamespace: true, labelmultihull.dev/managed-by: multihull,helm.sh/resource-policy: keep) unless it equals the release namespace. If amultihullNamespace already exists outside Helm, the upgrade fails to adopt it: setworkloads.createNamespace=falseorworkloads.namespaceto another name.helm uninstallleaves the Namespace and the workloads in it. edge_errorsnapshot field.Endpoint.edge_erroris optional indiscovery.protoand the snapshot JSON. A 0.2.0 router without it behaves as 0.1.0 did; a 0.1.0 router ignores it. Mixed versions of controller and router work either way, and the Modal translator sets it on every new snapshot.- Images. The
0.2.0tags are manifest lists forlinux/amd64andlinux/arm64. Local builds andkind loadworkarounds for Apple Silicon are no longer needed;charts/multihull/Chart.yamllists both platforms underartifacthub.io/images. hull deployis a dry run without--apply, as it was in 0.1.0; the docs now say so.
Pull requests
- docs: record the 0.1.0 release by @mishraprafful in #125
- feat(chart): prepare the Helm chart for an Artifact Hub listing by @mishraprafful in #126
- ci(secrets): scan every PR and push to main with gitleaks by @mishraprafful in #129
- feat(website): add a docs version switcher by @mishraprafful in #128
- docs(handover): keep session entries in one file per session by @mishraprafful in #133
- fix: resolve the CodeQL alerts on main by @mishraprafful in #132
- feat(cli): add hull top, a live view of router endpoints and routes by @mishraprafful in #131
- feat(demo): add make demo, the local failover demo by @mishraprafful in #130
- docs: drop the GHCR token steps now that the packages are public by @mishraprafful in #135
- docs(handover): keep the handover short and turn the session log into lessons by @mishraprafful in #134
- feat(demo): add make demo-cloud, the kind and Modal failover demo by @mishraprafful in #136
- feat(demo): record the local demo with vhs and embed it in README and docs by @mishraprafful in #137
- docs(security): state the supported versions now that 0.1.0 is released by @mishraprafful in #144
- docs: launch checklist, badges, code of conduct and milestone names by @mishraprafful in #143
- ci: pull base images from Docker Hub mirrors pinned by digest by @mishraprafful in #142
- feat(cli): add --version by @mishraprafful in #138
- fix(router): fail over on provider edge 404s instead of returning them by @mishraprafful in #146
- chore(launch): add the social preview image and record a cloud demo run by @mishraprafful in #153
- test(e2e): wait for scale attempts on every target before comparing them by @mishraprafful in #151
- fix(kubernetes): standardise on the multihull namespace owned by the chart by @mishraprafful in #154
- docs(quickstart): fix stumbles from a fresh-user walkthrough of main by @mishraprafful in #155
- ci(images): publish linux/amd64 and linux/arm64 manifests by @mishraprafful in #156
- chore(release): prepare 0.2.0 by @mishraprafful in #157
Full Changelog: v0.1.0...v0.2.0
- docs: recor...
v0.1.0
What's Changed
- docs: add Multihull architecture plan by @mishraprafful in #1
- build: add package skeletons to reserve multihull on PyPI and crates.io by @mishraprafful in #2
- feat: scaffold v0.1 control plane, router, proto, chart, example and docs site by @mishraprafful in #3
- ci: add GitHub Actions, release skeleton and generated reference docs by @mishraprafful in #5
- feat(python): add hull deploy, destroy, logs, controller and sdk by @mishraprafful in #6
- feat(router): wire sticky routing, provider circuits, adaptive concurrency, TLS and http snapshot source by @mishraprafful in #7
- docs: update handover after merging PRs 5 to 7 by @mishraprafful in #8
- ci: deploy docs to Cloudflare Pages by @mishraprafful in #16
- ci: delete docs preview deployments when a PR closes by @mishraprafful in #17
- ci: sweep docs preview deployments older than 24 hours daily by @mishraprafful in #19
- docs: add end-to-end testing strategy by @mishraprafful in #18
- docs: add contributing guide by @mishraprafful in #21
- feat(testing): add mock model server fixture by @mishraprafful in #23
- feat(python): add docker provider and make gpu optional for docker-only targets by @mishraprafful in #24
- feat(website): add animated failover hero and spec fan-out illustrations by @mishraprafful in #22
- feat(testing): add local end-to-end harness and fix the failover bugs it found by @mishraprafful in #25
- feat(website): minimal refresh of the docs site by @mishraprafful in #26
- docs: align retry budget and disconnect docs with the code, refresh handover by @mishraprafful in #27
- feat(router): close the four gaps found by the end-to-end harness by @mishraprafful in #29
- fix: resolve the four live-run blockers from the PR 29 review by @mishraprafful in #30
- chore: point package and chart homepages at multihull.pages.dev by @mishraprafful in #32
- ci: move GitHub Actions to Node 24 releases by @mishraprafful in #34
- fix(router): stop probes from masking a failing model by @mishraprafful in #33
- feat: prepare the first live run with kind in CI and a Modal smoke workflow by @mishraprafful in #31
- fix(router): keep queue pressure until the admission queue drains, and stabilise e2e waits by @mishraprafful in #35
- ci: pin GitHub-hosted runners to ubuntu-24.04 by @mishraprafful in #36
- docs: add a security policy by @mishraprafful in #38
- ci(release): publish crates through trusted publishing instead of a registry token by @mishraprafful in #44
- chore(github): add issue forms and route vulnerability reports to the security policy by @mishraprafful in #78
- docs: note that crates.io releases also use trusted publishing by @mishraprafful in #79
- build(deps): bump http-cache-semantics from 4.2.0 to 4.3.0 in /website in the npm_and_yarn group across 1 directory by @dependabot[bot] in #52
- fix(github): remove duplicate security and docs entries from the issue chooser by @mishraprafful in #80
- fix(modal): make the first live smoke pass with credential checks and a pinned image builder by @mishraprafful in #37
- fix(router): validate health paths and tuning values, and update failover docs by @mishraprafful in #83
- fix(chart): render router.toml values by type and keep extraConfig keys top-level by @mishraprafful in #84
- fix(controller): keep raised floors safe across races, silence and restarts by @mishraprafful in #85
- fix(control-plane): hash route API keys the way the router verifies them by @mishraprafful in #82
- feat(website): restyle the docs site after the hunk.dev reference by @mishraprafful in #81
- fix(router): end streams at timeouts.total without blaming the endpoint, and reserve before the half-open trial by @mishraprafful in #86
- fix: fail closed when a route's API key source yields no keys by @mishraprafful in #89
- ci(release): publish the Helm chart to GHCR as an OCI artifact by @mishraprafful in #90
- feat(control-plane): secure the discovery stream with TLS, mTLS and a bootstrap token by @mishraprafful in #87
- fix(router): keep the TTFT baseline and re-send Degraded while a slowdown lasts by @mishraprafful in #96
- ci(release): grant publish-crates read access to repo contents and pin Helm in kind by @mishraprafful in #99
- test(e2e): isolate concurrent runs and stop test_02 requiring a transient failover by @mishraprafful in #97
- refactor(router): fold the internal crates into one published multihull crate by @mishraprafful in #100
- feat(control-plane): publish a controller image and read the state backend from MULTIHULL_STATE_BACKEND by @mishraprafful in #101
- build(deps): bump postcss-selector-parser to 7.1.6 in website by @mishraprafful in #110
- fix(control-plane): keep snapshot versions monotonic and never empty a serving route by @mishraprafful in #111
- fix(website): follow the system colour scheme by default instead of forcing dark by @mishraprafful in #113
- feat(live): GPU live run on two Modal L4 targets by @mishraprafful in #114
- docs(kubernetes): drop the Ingress and HTTPRoute claims and mark them as planned by @mishraprafful in #118
- docs(release): prepare the 0.1.0 release documentation by @mishraprafful in #115
- fix(router): retry a tried provider when no untried endpoint can take the request by @mishraprafful in #119
- fix(modal): make the live GPU run deploy and fail over on real vLLM containers by @mishraprafful in #120
- docs(handover): record the 0.1.0 release verification runs and findings by @mishraprafful in #121
- docs(changelog): date the 0.1.0 section for the release by @mishraprafful in #123
New Contributors
- @mishraprafful made their first contribution in #1
- @dependabot[bot] made their first contribution in #52
Full Changelog: https://github.com/mishraprafful/multihull/commits/v0.1.0
What's Changed
- docs: add Multihull architecture plan by @mishraprafful in #1
- build: add package skeletons to reserve multihull on PyPI and crates.io by @mishraprafful in #2
- feat: scaffold v0.1 control plane, router, proto, chart, example and docs site by @mishraprafful in #3
- ci: add GitHub Actions, release skeleton and generated reference docs by @mishraprafful in #5
- feat(python): add hull deploy, destroy, logs, controller and sdk by @mishraprafful in #6
- feat(router): wire sticky routing, provider circuits, adaptive concurrency, TLS and http snapshot source by @mishraprafful in #7
- docs: update handover after merging PRs 5 to 7 by @mishraprafful in #8
- ci: deploy docs to Cloudflare Pages by @mishraprafful in #16
- ci: delete docs preview deployments when a PR closes by @mishraprafful in #17
- ci: sweep docs preview deployments older than 24 hours daily by @mishraprafful in #19
- docs: add end-to-end testing str...