Skip to content

Releases

v0.2.0

Choose a tag to compare

@mishraprafful mishraprafful released this 12 Oct 10:57
d67e32d

Stable launch, tagged 2026-10-12. Multihull deploys one container to Kubernetes and Modal from one multihull.yaml, serves both behind one URL through a Rust router, and fails over between them per request. 0.2.0 adds the demos, hull top, multi-arch images, failover on provider edge errors and a quickstart that a fresh user can follow end to end.

Scope, plainly: a Kubernetes plus Modal failover router for a single model, with Docker as the free local layer. Not yet the five-provider framework the architecture plan describes. 0.1.0 notes: docs/releases/0.1.0.md.

Install

uv tool install 'multihull[modal]==0.2.0'
cargo install multihull --version 0.2.0
helm pull oci://ghcr.io/mishraprafful/charts/multihull --version 0.2.0

Images: ghcr.io/mishraprafful/multihull-router:0.2.0, multihull-controller:0.2.0, multihull-mock-server:0.2.0, each a linux/amd64 and linux/arm64 manifest list. The GitHub release carries router binaries for linux amd64, linux arm64 and macOS arm64. Images, chart and packages are public; see Install from OCI.

Highlights

  • make demo: the five-beat failover demo on three local docker targets, free, no GPUs, no credentials; recorded in the README. make demo-cloud runs the same beats on kind plus Modal for cents.
  • hull top: a live terminal view of the router, per endpoint and per route, from the admin listener.
  • hull --version, and a quickstart rewritten from a fresh-user walkthrough: seven steps through the first request, with the measured time.
  • Images for linux/amd64 and linux/arm64 under every tag, so an Apple Silicon kind node or Docker target pulls them directly.
  • The Kubernetes namespace is multihull: hull init writes it, the Helm chart creates it, hull deploy stops with one line when it is missing.
  • Provider edge errors fail over: a stopped Modal app answers 404 modal-http: invalid function call from the edge for about 14 s before the probe ejects it; the router now retries those on another provider, even for a POST without an Idempotency-Key. A model's own 404 is still returned.
  • Docs version switcher, Artifact Hub metadata on the chart, gitleaks on every PR, base images from Docker Hub mirrors pinned by digest.

What is proven

By live runs:

  • Cloud demo on 2026-10-10 (make demo-cloud, kind primary, Modal secondary, mock server on CPU, 180 s scripted): 511 requests, 0 client errors, 139 failovers. Deploy to both Ready in 14 s with the Modal image cached. The primary scaled to zero at 40 s, the circuit opened 12 s later and the probe marked it down 5 s after that with every request in between served by Modal; the restore at 70 s reached half_open at 94 s and closed at 99 s, then traffic returned to kind. Modal bill 0.0012 USD. Recorded in docs/runbooks/demo-cloud.md.
  • A real model on GPUs: two consecutive live GPU runs on main on 2026-10-08 (37821488815, 37822124427, live-gpu.yml) deployed Qwen/Qwen2.5-1.5B-Instruct on vLLM 0.11.0 to two Modal L4 targets, streamed tokens over SSE and plain completions, stopped the primary app and served every failover request from the secondary with zero client errors (23 requests per run). Deploy reached Ready in about 100 s with the image cached. These runs predate the edge-404 fix; the failover test now asserts zero 404s after modal app stop and waits for an owner run.
  • Fresh-user walkthrough of the quickstart on macOS arm64 against a kind cluster: six minutes of wall clock, cluster creation and a router build included; make demo from a clean clone in 73 s (182 requests, 0 errors, 91 failovers). The chart was installed from the published OCI package.

On every PR and push to main:

  • The local e2e suite, 36 rows in about 4 minutes; the kind suite, 51 rows (kind primary, docker secondary); the in-cluster suite, 9 rows (controller and router from the chart in kind over mTLS, controller restarts with kept and wiped state); the scripted make demo, 60 s with a kill and restore and zero client errors.
  • Unit and golden tests for the control plane (361 tests) and the router.

Not proven:

  • Kubernetes with a GPU: kind has no GPUs, so node selectors and tolerations have never scheduled on a real cluster, and no real model has run on Kubernetes.
  • A cluster pull from a private registry.
  • RunPod, Baseten and Replicate are render-only: hull plan works, apply raises not implemented.
  • Only SQLite state exists. s3://, gs:// and postgres:// exit with a not-implemented error.
  • No sustained load or multi-replica router measurements. The plan's latency and throughput numbers are targets, not results.
  • The chart ships no RBAC for the controller.
  • No outside user has run hull init through hull deploy; the walkthrough above was done by a maintainer.

Most of this code was written by coding agents and reviewed by CI and other agents. A human read-through of the router's retry and circuit paths (router/crates/multihull/src/proxy/handler.rs, core/circuit.rs, core/retry.rs) is recommended before relying on it in production.

Upgrade notes

From 0.1.0:

  • Namespace. hull init now writes namespace: multihull; a spec written by 0.1.0 keeps whatever it has (inference from the old hull init, or default when unset) and keeps deploying there. hull deploy fails before applying when the target namespace does not exist; create it, or install the chart, which creates multihull.
  • Chart workloads.* values. Chart 0.2.0 renders a multihull Namespace (workloads.namespace, workloads.createNamespace: true, label multihull.dev/managed-by: multihull, helm.sh/resource-policy: keep) unless it equals the release namespace. If a multihull Namespace already exists outside Helm, the upgrade fails to adopt it: set workloads.createNamespace=false or workloads.namespace to another name. helm uninstall leaves the Namespace and the workloads in it.
  • edge_error snapshot field. Endpoint.edge_error is optional in discovery.proto and the snapshot JSON. A 0.2.0 router without it behaves as 0.1.0 did; a 0.1.0 router ignores it. Mixed versions of controller and router work either way, and the Modal translator sets it on every new snapshot.
  • Images. The 0.2.0 tags are manifest lists for linux/amd64 and linux/arm64. Local builds and kind load workarounds for Apple Silicon are no longer needed; charts/multihull/Chart.yaml lists both platforms under artifacthub.io/images.
  • hull deploy is a dry run without --apply, as it was in 0.1.0; the docs now say so.

Pull requests

Full Changelog: v0.1.0...v0.2.0

  • docs: recor...
Read more

v0.1.0

v0.1.0 Pre-release
Pre-release

Choose a tag to compare

@mishraprafful mishraprafful released this 09 Oct 06:54
4a60f83

What's Changed

  • docs: add Multihull architecture plan by @mishraprafful in #1
  • build: add package skeletons to reserve multihull on PyPI and crates.io by @mishraprafful in #2
  • feat: scaffold v0.1 control plane, router, proto, chart, example and docs site by @mishraprafful in #3
  • ci: add GitHub Actions, release skeleton and generated reference docs by @mishraprafful in #5
  • feat(python): add hull deploy, destroy, logs, controller and sdk by @mishraprafful in #6
  • feat(router): wire sticky routing, provider circuits, adaptive concurrency, TLS and http snapshot source by @mishraprafful in #7
  • docs: update handover after merging PRs 5 to 7 by @mishraprafful in #8
  • ci: deploy docs to Cloudflare Pages by @mishraprafful in #16
  • ci: delete docs preview deployments when a PR closes by @mishraprafful in #17
  • ci: sweep docs preview deployments older than 24 hours daily by @mishraprafful in #19
  • docs: add end-to-end testing strategy by @mishraprafful in #18
  • docs: add contributing guide by @mishraprafful in #21
  • feat(testing): add mock model server fixture by @mishraprafful in #23
  • feat(python): add docker provider and make gpu optional for docker-only targets by @mishraprafful in #24
  • feat(website): add animated failover hero and spec fan-out illustrations by @mishraprafful in #22
  • feat(testing): add local end-to-end harness and fix the failover bugs it found by @mishraprafful in #25
  • feat(website): minimal refresh of the docs site by @mishraprafful in #26
  • docs: align retry budget and disconnect docs with the code, refresh handover by @mishraprafful in #27
  • feat(router): close the four gaps found by the end-to-end harness by @mishraprafful in #29
  • fix: resolve the four live-run blockers from the PR 29 review by @mishraprafful in #30
  • chore: point package and chart homepages at multihull.pages.dev by @mishraprafful in #32
  • ci: move GitHub Actions to Node 24 releases by @mishraprafful in #34
  • fix(router): stop probes from masking a failing model by @mishraprafful in #33
  • feat: prepare the first live run with kind in CI and a Modal smoke workflow by @mishraprafful in #31
  • fix(router): keep queue pressure until the admission queue drains, and stabilise e2e waits by @mishraprafful in #35
  • ci: pin GitHub-hosted runners to ubuntu-24.04 by @mishraprafful in #36
  • docs: add a security policy by @mishraprafful in #38
  • ci(release): publish crates through trusted publishing instead of a registry token by @mishraprafful in #44
  • chore(github): add issue forms and route vulnerability reports to the security policy by @mishraprafful in #78
  • docs: note that crates.io releases also use trusted publishing by @mishraprafful in #79
  • build(deps): bump http-cache-semantics from 4.2.0 to 4.3.0 in /website in the npm_and_yarn group across 1 directory by @dependabot[bot] in #52
  • fix(github): remove duplicate security and docs entries from the issue chooser by @mishraprafful in #80
  • fix(modal): make the first live smoke pass with credential checks and a pinned image builder by @mishraprafful in #37
  • fix(router): validate health paths and tuning values, and update failover docs by @mishraprafful in #83
  • fix(chart): render router.toml values by type and keep extraConfig keys top-level by @mishraprafful in #84
  • fix(controller): keep raised floors safe across races, silence and restarts by @mishraprafful in #85
  • fix(control-plane): hash route API keys the way the router verifies them by @mishraprafful in #82
  • feat(website): restyle the docs site after the hunk.dev reference by @mishraprafful in #81
  • fix(router): end streams at timeouts.total without blaming the endpoint, and reserve before the half-open trial by @mishraprafful in #86
  • fix: fail closed when a route's API key source yields no keys by @mishraprafful in #89
  • ci(release): publish the Helm chart to GHCR as an OCI artifact by @mishraprafful in #90
  • feat(control-plane): secure the discovery stream with TLS, mTLS and a bootstrap token by @mishraprafful in #87
  • fix(router): keep the TTFT baseline and re-send Degraded while a slowdown lasts by @mishraprafful in #96
  • ci(release): grant publish-crates read access to repo contents and pin Helm in kind by @mishraprafful in #99
  • test(e2e): isolate concurrent runs and stop test_02 requiring a transient failover by @mishraprafful in #97
  • refactor(router): fold the internal crates into one published multihull crate by @mishraprafful in #100
  • feat(control-plane): publish a controller image and read the state backend from MULTIHULL_STATE_BACKEND by @mishraprafful in #101
  • build(deps): bump postcss-selector-parser to 7.1.6 in website by @mishraprafful in #110
  • fix(control-plane): keep snapshot versions monotonic and never empty a serving route by @mishraprafful in #111
  • fix(website): follow the system colour scheme by default instead of forcing dark by @mishraprafful in #113
  • feat(live): GPU live run on two Modal L4 targets by @mishraprafful in #114
  • docs(kubernetes): drop the Ingress and HTTPRoute claims and mark them as planned by @mishraprafful in #118
  • docs(release): prepare the 0.1.0 release documentation by @mishraprafful in #115
  • fix(router): retry a tried provider when no untried endpoint can take the request by @mishraprafful in #119
  • fix(modal): make the live GPU run deploy and fail over on real vLLM containers by @mishraprafful in #120
  • docs(handover): record the 0.1.0 release verification runs and findings by @mishraprafful in #121
  • docs(changelog): date the 0.1.0 section for the release by @mishraprafful in #123

New Contributors

Full Changelog: https://github.com/mishraprafful/multihull/commits/v0.1.0

What's Changed

  • docs: add Multihull architecture plan by @mishraprafful in #1
  • build: add package skeletons to reserve multihull on PyPI and crates.io by @mishraprafful in #2
  • feat: scaffold v0.1 control plane, router, proto, chart, example and docs site by @mishraprafful in #3
  • ci: add GitHub Actions, release skeleton and generated reference docs by @mishraprafful in #5
  • feat(python): add hull deploy, destroy, logs, controller and sdk by @mishraprafful in #6
  • feat(router): wire sticky routing, provider circuits, adaptive concurrency, TLS and http snapshot source by @mishraprafful in #7
  • docs: update handover after merging PRs 5 to 7 by @mishraprafful in #8
  • ci: deploy docs to Cloudflare Pages by @mishraprafful in #16
  • ci: delete docs preview deployments when a PR closes by @mishraprafful in #17
  • ci: sweep docs preview deployments older than 24 hours daily by @mishraprafful in #19
  • docs: add end-to-end testing str...
Read more