v0.5.0

Modelplane v0.5.0

The theme of this release is the control plane's edge and what it can tell you. The gateway in front of your fleet became an AI gateway, and every cluster in the fleet became observable.

InferenceGateway is now an Envoy AI Gateway, the same one each InferenceCluster already runs. It authenticates callers and reads the model a request names in its body, then resolves the ModelService serving it. For the backend it picks, it rewrites the model name, the credential and the path. ModelService gains priority beside weight, so traffic splits within a priority and fails over to the next. A gateway is no longer a singleton either: it names the cluster it runs on, so a fleet can run one per region.

Modelplane now collects the fleet's metrics. An OpenTelemetry collector runs on every inference cluster. It renames each component's series into one modelplane_* vocabulary and exports wherever a TelemetryDestination says. Because the engine is your choice, the names are not Modelplane's to dictate, so a MetricMapping is what turns an engine's own vocabulary into the shared one. vLLM, SGLang, the gateway, the endpoint picker and DCGM are mapped for you.

Civo joins EKS, AKS, GKE, Nebius and Vultr as a cloud Modelplane can provision an InferenceCluster on, and a ModelDeployment can now scale to zero replicas and park without looking broken.

Check out https://docs.modelplane.ai/getting-started/ to get started, and the v0.5 docs for the rest.

Highlights

• Make InferenceGateway an AI gateway by @negz in #425
• Collect metrics from every inference cluster by @dennis-upbound in #470
• Add Civo as an InferenceCluster provider by @haarchri in #475
• Allow scaling a ModelDeployment to zero replicas by @haarchri in #454
• Block InferenceCluster deletion while anything composes onto it by @negz in #465
• Place cached replicas only on clusters with cache storage by @negz in #466

What's Changed

• Point the docs dropdown and apex at v0.4 by @negz in #442
• Remove 0.1 from main by @tr0njavolta in #445
• Document the per-cluster serving stack choice by @negz in #448
• Add a guide for multi-node serving on Dynamo by @negz in #450
• Docs: Move Hugo engine out of this repo by @tr0njavolta in #433
• Bump the e2e workload node image past the DRA idle deadlock by @nicknikolakakis in #421
• Docs: Updates the PR comment step in the docs previews. by @tr0njavolta in #460
• Docs: Drop the per-cloud storage list from the ModelCache page by @haarchri in #452
• Add install and existing-cluster requirements to the docs by @negz in #458
• Link the Install page from the docs sidebar by @negz in #463
• Make InferenceGateway an AI gateway by @negz in #425
• Design: managed metrics collection by @dennis-upbound in #363
• Ignore coverage artifacts by @dennis-upbound in #471
• Allow scaling a ModelDeployment to zero replicas by @haarchri in #454
• Stop serving stack pods tolerating every taint by @haarchri in #456
• Place cached replicas only on clusters with cache storage by @negz in #466
• Fix the ModelCache printer columns, example manifests and Dynamo guide names by @negz in #469
• Qualify the ModelExpress address with the server's namespace by @negz in #464
• Mark XRs not ready while waiting, and replicas until each object is by @negz in #467
• Block InferenceCluster deletion while anything composes onto it by @negz in #465
• Bump the AWS providers to v2.8.1 by @ldalorion in #481
• Add Civo as an InferenceCluster provider by @haarchri in #475
• Write the image's passwd and group as files, not symlinks by @dennis-upbound in #487
• fix(ci): skip docs job in forks by @haarchri in #484
• feat(bump): nebius update to v1.2.0 by @haarchri in #485
• Collect metrics from every inference cluster by @dennis-upbound in #470

New Contributors

• @nicknikolakakis made their first contribution in #421
• @ldalorion made their first contribution in #481

Full Changelog: v0.4.0...v0.5.0