Skip to content

OME 1.2.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 14 Jul 23:32
· 157 commits to main since this release
v1.2.0
29bab9f
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

OME 1.2.0

1.2.0 is about where model weights come from and what's still in the API. Pre-provisioned and air-gapped weights on shared storage become a first-class model source, the long-deprecated predictor field is finally removed, and the model-agent's task handling gets meaningfully sturdier under deletion, reuse, and churn.

✨ Highlights

PVC-Backed BaseModels (#653, #654, #655, #656, #657, #658, #659, #668)

Until now a pvc:// BaseModel could be parsed but never acted on — nothing is downloaded for a PVC model, so it sat in Importing/In_Transit forever while the controller waited on per-node status ConfigMaps that never arrived. This release closes that gap and makes PVC a first-class, Ready-reaching model source:

  • Serve from shared storage — point a BaseModel or ClusterBaseModel at pvc://… (CSI, NFS, Lustre) and it reaches Ready with no per-node download and no egress — ideal for pre-provisioned or air-gapped weights
  • On-demand metadata — a short-lived ome-agent model-metadata Job parses the model on the claim and publishes a per-model status ConfigMap the controller reconciles, instead of writing the CR directly (#668)
  • Mounted read-only into serving pods — the claim is mounted at the model path with the correct sub-path; the per-node model-ready affinity is skipped since PVCs aren't node-scoped (#656)
  • Validated at admission — a validating webhook rejects malformed or wrongly-scoped pvc:// URIs at kubectl apply time instead of surfacing later as a Failed status (#657, #658)
  • Condition-driven status — model status gains Conditions, ObservedGeneration, LastReconcileTime, and SourceReachable/Ready conditions with PVC-lifecycle reasons (PVCNotBound, MetadataExtracting, Validated, ExtractionFailed, …) (#653)
  • Backend registry seam — the BaseModel controller is now a thin dispatcher over pluggable Backends, so the PVC path stays fully isolated from the per-node download path (#654)

Namespacing: a namespaced BaseModel uses pvc://{claim}/{sub-path} (its own namespace is implied); a ClusterBaseModel must scope the claim explicitly with pvc://{namespace}:{claim}/{sub-path}.

The Deprecated predictor Field Is Removed (#661)

  • spec.predictor and the entire PredictorSpec type are gone — serving is now configured exclusively via spec.engine + spec.model (plus spec.decoder / spec.router for disaggregated setups)
  • Removes ~22k lines, the bulk being the predictor schema stripped from the InferenceService CRD and the regenerated OpenAPI / deepcopy / defaults
  • Breaking: applying any manifest that still contains predictor now fails (unknown field "spec.predictor"); GitOps/CI syncs of stale manifests hard-fail until predictor is removed from source
  • No disruption to running services — a stale predictor: {} left on an existing object is pruned automatically on the next write (a reconcile, kubectl edit, even kubectl annotate); no manual object cleanup required

Sturdier Model-Agent Task Handling (#650, #660, #663)

  • Deletion and reuse no longer wait behind downloads — a dedicated high-priority worker path runs lightweight delete and same-path reuse/wait work so it isn't blocked behind long-running download/validation; deletion state is re-checked before starting work and before writing Ready, so a stale queued download can't mark an already-deleted model Ready (#650)
  • Metadata-only spec updates stop forcing re-downloads — parser-populated fields (model type, framework, format, capabilities) no longer trigger a DownloadOverride; storage, label/annotation, and TensorRT-LLM routing changes still do (#660)
  • No more reconcile churn on modelConfiguration — a sorted-key vs declaration-order byte mismatch made the agent rewrite the BaseModel spec on every reconcile; the field is now preserve-if-unset like every other field (#663)

⚠️ Upgrading:

  • Remove predictor from your manifest files. spec.predictor no longer exists in the schema, so any kubectl apply or GitOps sync of a file still containing a predictor: block will hard-fail. Running services self-heal automatically — only the source files need editing (#661).
  • PVC support needs new RBAC and webhooks. The metadata-extraction Job writes a status ConfigMap in the OME namespace, and the BaseModel/ClusterBaseModel validating webhooks must be wired with CA injection. The bundled charts and manifests include both grants (#659, #668) — if you manage RBAC or webhooks out of band, add them before using pvc:// models.

🚀 Features & Enhancements

  • PVC-backed BaseModel/ClusterBaseModel as a first-class, Ready-reaching model source, backed by a pluggable backend registry (#654, #655, #656)
  • pvc:// URI validation plus a validating admission webhook for BaseModel and ClusterBaseModel (#657, #658)
  • Model status conditions — Conditions, ObservedGeneration, LastReconcileTime, SourceReachable/Ready — with PVC-lifecycle reason constants (#653)
  • ome-agent model-metadata now publishes a per-model status ConfigMap instead of writing the CR directly, fixing cross-namespace ClusterBaseModel Jobs and models stuck at In_Transit (#668)
  • Deployment-mode-aware runtime auto-selection — engine/decoder ome.io/deploymentMode annotations are compared between the InferenceService and the runtime, so single-node and multi-node runtimes for the same model are distinguished; missing/empty/invalid values stay unset and add no constraint (#647)
  • High-priority model-agent worker path for lightweight delete and same-path reuse tasks (#650)
  • Metadata-only BaseModel spec updates no longer trigger DownloadOverride re-downloads (#660)
  • B300 added as a compatible OCI instance type in the model agent (#669)

🐛 Bug Fixes

  • Preserve spec.modelConfiguration unless it is unset, ending a per-reconcile spec rewrite caused by a sorted-key vs declaration-order byte mismatch (#663)
  • Truncate the LWS leader-pod app label to 63 characters and cap generated LWS names at 35 characters, so long component names no longer violate Kubernetes label-value limits (#649)
  • Wire the four missing validating webhooks (including BaseModel/ClusterBaseModel) and their CA injection in the kustomize config/default overlay, bringing it to parity with the Helm chart (#659)

♻️ Maintenance

  • New pre-commit hook keeps web-console/frontend/package-lock.json in sync with package.json, so the npm ci step in CI can't drift on a stale lockfile; documented in CONTRIBUTING.md (#667)

📦 Container Images

The following container images are available:

# OME Manager
docker pull ghcr.io/moirai-internal/ome-manager:v1.2.0

# Model Agent
docker pull ghcr.io/moirai-internal/model-agent:v1.2.0

# OME Agent
docker pull ghcr.io/moirai-internal/ome-agent:v1.2.0

# Multinode Prober
docker pull ghcr.io/moirai-internal/multinode-prober:v1.2.0

⎈ Helm Installation

Option 1: OCI Registry (Recommended)

# Install directly from OCI registry
helm install ome-crd oci://ghcr.io/moirai-internal/charts/ome-crd --version 1.2.0 --namespace ome --create-namespace
helm install ome oci://ghcr.io/moirai-internal/charts/ome-resources --version 1.2.0 --namespace ome

Option 2: GitHub Releases

# Add the OME Helm repository
helm repo add ome https://github.com/ome-projects/ome/releases/download/v1.2.0
helm repo update

# Install OME
helm install ome-crd ome/ome-crd --namespace ome --create-namespace
helm install ome ome/ome-resources --namespace ome

📋 Changelog

Full Changelog: v1.1.0...v1.2.0

📄 Software Bill of Materials

SBOMs are available in both SPDX and CycloneDX formats for:

  • Source code
  • All container images

🔐 Signatures

All container images are signed with cosign. Verify with:

cosign verify ghcr.io/moirai-internal/ome-manager:v1.2.0   --certificate-identity-regexp=https://github.com/ome-projects/ome/.github/workflows/release.yaml@refs/tags/.*   --certificate-oidc-issuer=https://token.actions.githubusercontent.com

What's Changed

New Contributors

Contributors

davidsnahm, pallasathena92, and 2 other contributors
Assets 21
Loading