Skip to content

v0.7.0

Latest
Latest

Choose a tag to compare

View all tags
@negz negz released this 09 Nov 23:40
· 2 commits to release-0.7 since this release
v0.7.0
9eb9487
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Modelplane v0.7.0

Modelplane now provisions inference clusters on CoreWeave. Set an InferenceCluster's source to CoreWeave and Modelplane creates a CoreWeave Kubernetes Service cluster with its VPC and node pools, then installs the serving stack. A ModelDeployment runs there unchanged. Thanks to CoreWeave for the GPU capacity we built and tested this on, and to their engineers for working through the bugs with us.

v0.7 also fixes ways a model could stop serving. A Standalone engine's rollout used to start a new pod before stopping an old one, which deadlocks on a full GPU pool. It now replaces one copy at a time, so an engine with one copy stops serving while its replacement starts. Each replica used to run one endpoint picker, and the cluster's gateway won't route without it, so draining the picker's node took the model offline. Replicas now run two pickers, spread across nodes where there's room, and a drain evicts only one at a time. The cluster's gateway proxy is still one pod, so a drain can still interrupt serving. A new InferenceGateway no longer returns HTTP 500s for its first model.

On EKS, AKS, Civo, Nebius and CoreWeave, GPU metrics now have the deployment, replica, role and engine labels of the pod using the GPU, so you can join them to that deployment's engine metrics.

Modelplane used to place replicas on AMD GPU pools a whole node at a time. Now small models can share AMD nodes, as they already could NVIDIA nodes.

Check out https://docs.modelplane.ai/getting-started/ to get started, and the v0.7 docs for the rest.

To try Modelplane without a cloud account, @tr0njavolta's local demo runs it on your laptop with fake GPUs and models small enough for a CPU.

Highlights

What's Changed

  • Attribute GPU series to the pod that holds the GPU by @dennis-upbound in #613
  • Review pull requests with an agent in CI by @negz in #611
  • Keep an AI Gateway route attached by @mikemikimike in #538
  • Let the review workflow check out a fork's PR by @negz in #627
  • Stop an engine's old copy before starting its replacement by @dennis-upbound in #572
  • Hold an engine Deployment unready until its rollout finishes by @dennis-upbound in #570
  • Reject a model member without template.spec at admission by @dennis-upbound in #577
  • Key every engine workload by its backend by @dennis-upbound in #576
  • Give the cluster functions' Helm releases fixed names by @humoflife in #587
  • Run two endpoint pickers with a disruption budget by @dennis-upbound in #575
  • Expect the endpoint picker's PodDisruptionBudget in AgentNotReporting by @haarchri in #643
  • ServingStack: Stop installing the Prometheus stack on inference clusters by @haarchri in #624
  • Fix the getting started dead ends for Vultr and Civo, and the VKE system pool plan VKE rejects by @haarchri in #615
  • Taint GKE GPU node pools explicitly by @negz in #636
  • Lower the CPU requests that strand an nvsentinel pod by @negz in #641
  • Stop pinning the install guide's kind node image by @negz in #634
  • Run self-hosted Renovate to bump crossplane-project.yaml dependencies, ci workflows by @haarchri in #592
  • Create a local quickstart using synthetic GPU by @tr0njavolta in #620
  • Pin dependencies by @modelplane-renovate[bot] in #646
  • Update dependency crossplane/crossplane to v2.3.7 by @modelplane-renovate[bot] in #649
  • Update nixpkgs-unstable digest to 7442dc2 by @modelplane-renovate[bot] in #647
  • Update dependency renovate to v44.216.5 by @modelplane-renovate[bot] in #652
  • Observe AMD GPU and CPU pools with the cluster agent by @negz in #650
  • Stop documenting metrics and merges the collector doesn't produce by @negz in #639
  • Bump oauthlib from 4.0.0 to 4.0.1 in the uv group across 1 directory by @dependabot[bot] in #656
  • Update nixpkgs-unstable digest to 0bbd846 by @modelplane-renovate[bot] in #651
  • Add a Kimi K3.1 Pro recipe on one 8x B300 node by @dennis-upbound in #560
  • Add the Modelplane charter by @bassam in #659
  • Update dependency renovate to v44.221.4 by @modelplane-renovate[bot] in #658
  • Let sensitive PRs opt out of the automatic review by @negz in #661
  • Adopt formal governance for the project by @bassam in #662
  • Update the commit and PR examples in CONTRIBUTING.md by @pluna in #663
  • Track branches, not commits, in the review VM flake's inputs by @negz in #660
  • Provision InferenceClusters on CoreWeave by @negz in #655

New Contributors

Full Changelog: v0.6.0...v0.7.0

Contributors

pluna, negz, and 7 other contributors
Assets 2
Loading