v0.6.0

Modelplane v0.6.0

GPUs are the most expensive part of an inference fleet. Modelplane used to place replicas a whole node at a time, so a model that needed one GPU on an eight-GPU node left seven idle. Small models can now share a node, maximizing GPU utilization.

Modelplane now serves on AMD Instinct GPUs as well as NVIDIA. Bring a cluster with AMD GPUs and Modelplane schedules models onto it through the same APIs, so you can run both vendors in one fleet. You can also try Modelplane without any GPUs. A small model can run on CPUs on any cloud Modelplane provisions. New recipes run Qwen2.5-0.5B on an AMD MI325X and on CPUs.

Disaggregated prefill/decode serving now works with SGLang. Modelplane used to write the router that pairs prefill and decode, and that router only understood vLLM. You now write it the way you write your engines: you pick the endpoint picker and sidecar images and set their flags and configuration, and Modelplane wires them into the cluster.

Check out https://docs.modelplane.ai/getting-started/ to get started, and the v0.6 docs for the rest.

If you're a platform engineer new to inference, @AnnieTalvasto's field guide to inference engineering covers the concepts, production operations and tooling you'll meet.

Highlights

• Schedule replicas by free GPUs rather than free nodes by @negz in #542
• Install each device stack only where its driver is claimed, adding CPU pools and AMD serving by @haarchri in #556
• Back node pools with CapacityReservations instead of capacityBlock by @negz in #600
• Compose the router the user writes for prefill/decode serving by @dennis-upbound in #529
• Route to a replica while it serves, and keep Ready for full capacity by @dennis-upbound in #562
• Install a DCGM exporter on Nebius clusters by @ldalorion in #584

What's Changed

• Pin the install guide to v0.5.0 by @dennis-upbound in #492
• Correct what the telemetry guide says about egress and credentials by @dennis-upbound in #504
• Bump vale-ai-tells to v1.37.0 by @negz in #507
• Port the unit and end-to-end tests to pytest by @negz in #482
• Update provider-kubernetes to 1.4.1 by @RedbackThomson in #513
• Declare AKS's default max surge on every node pool by @ytsarev in #516
• Assert what the served endpoints return, not just a 200 by @nicknikolakakis in #420
• Query the gateway's series by model in the telemetry docs by @ytsarev in #518
• Enable the DCGM exporter on Civo, document it for existing clusters by @haarchri in #526
• Map SGLang's prefix cache and retraction counters by @dennis-upbound in #544
• Design: multiple accelerator vendors in the serving stack by @haarchri in #432
• Count the requests the fleet gateway has open to each model by @dennis-upbound in #548
• Give a replica's remote objects names Kubernetes accepts by @negz in #468
• Fix wrong and empty built-in engine metrics by @dennis-upbound in #546
• Propose user-authored routing for prefill/decode serving by @dennis-upbound in #514
• Compose the router the user writes for prefill/decode serving by @dennis-upbound in #529
• Exercise each clouds serving stack placement in the local e2e by @haarchri in #472
• Report a stored prefill/decode pair's state when its routing is missing by @dennis-upbound in #555
• Pin the model cache hydration image to one that distrusts eMudhra by @negz in #561
• Render the picker's prefix match length explicitly by @dennis-upbound in #563
• Route to a replica while it serves, and keep Ready for full capacity by @dennis-upbound in #562
• ServingStack: Bump the pinned aicr to 1.0.0 by @haarchri in #539
• Drop Moonshot AI from the recipe catalog's vendors by @dennis-upbound in #559
• Project hygiene and cleanup - Add MAINTAINERS and ROADMAP by @jbw976 in #566
• Allow nvsentinel's NIC health monitor on the AKS GPU pool by @negz in #571
• Print the dev shell banner only in interactive shells by @negz in #590
• Schedule replicas by free GPUs rather than free nodes by @negz in #542
• Add field guide for platform engineers moving to inference by @AnnieTalvasto in #588
• Install each device stack only where its driver is claimed, adding CPU pools and AMD serving by @haarchri in #556
• Render observed subnet and filesystem ids into Nebius node groups by @haarchri in #593
• Install a DCGM exporter on Nebius clusters by @ldalorion in #584
• Bump providers to releases built with Go's eMudhra CA distrust by @haarchri in #591
• Document GPU exporter and gateway metric limits in telemetry docs by @tr0njavolta in #540
• Fit the AKS DNS prefix within Azure's 54 character limit by @dennis-upbound in #568
• Back node pools with CapacityReservations instead of capacityBlock by @negz in #600
• Fix stale docs ahead of v0.6 by @negz in #608
• Bump certifi from 2026.7.22 to 2026.10.18 in the uv group across 1 directory by @dependabot[bot] in #602

New Contributors

• @RedbackThomson made their first contribution in #513
• @jbw976 made their first contribution in #566
• @AnnieTalvasto made their first contribution in #588

Full Changelog: v0.5.0...v0.6.0