Summary
ModelDeployment accepts member-specific pod metadata at:
spec:
template:
spec:
engines:
- name: example
members:
- role: Leader
template:
metadata:
labels:
example.com/role: leader
annotations:
example.com/config: leader
spec:
containers:
- name: engine
image: example/engine:latest
- role: Worker
worker:
nodes: 1
template:
metadata:
labels:
example.com/role: worker
annotations:
example.com/config: worker
spec:
containers:
- name: engine
image: example/engine:latest
These fields round-trip into the resolved ModelReplica, but the workload composers do not merge them into the generated pod templates. As a result, the labels and annotations are silently ignored.
Current behavior
- The llm-d backend constructs leader metadata only from Modelplane-managed labels and constructs the worker pod template without metadata:
|
|
|
# Only the leader serves the OpenAI API → it carries the serving label |
|
# the replica's shared Service selects on, plus the role label, the |
|
# serving port, and the readiness probe. |
|
leader_pod = { |
|
"metadata": {"labels": {base.LABEL_SERVING: serving_label, _LABEL_ROLE: "leader"}}, |
|
"spec": pod_spec(leader, container(leader, serving=True)), |
|
} |
|
# The worker followers don't serve the OpenAI API, so they carry no |
|
# serving label - the replica's Service must never route to them. LWS |
|
# manages their gang membership labels itself. |
|
worker_pod = { |
|
"spec": pod_spec(worker, container(worker, serving=False)), |
|
} |
- The native backend likewise constructs pod metadata only from Modelplane-managed labels:
|
if tmpl.spec.imagePullSecrets: |
|
pod_spec["imagePullSecrets"] = [s.model_dump(exclude_none=True) for s in tmpl.spec.imagePullSecrets] |
|
|
|
deployment = { |
|
"apiVersion": "apps/v1", |
|
"kind": "Deployment", |
|
"metadata": {"name": name, "namespace": base.REMOTE_NAMESPACE}, |
|
"spec": { |
- The API explicitly exposes
template.metadata.labels and template.metadata.annotations:
|
template: |
|
type: object |
|
description: >- |
|
Pod template for this member's engine pods. A |
|
curated subset of PodTemplateSpec. |
|
properties: |
|
metadata: |
|
type: object |
|
description: >- |
|
Metadata applied to the member's pods. Useful |
|
for labels and annotations that control |
|
cluster-level features like service mesh |
|
injection. |
|
properties: |
|
labels: |
|
type: object |
|
additionalProperties: |
|
type: string |
|
annotations: |
|
type: object |
|
additionalProperties: |
|
type: string |
|
spec: |
|
type: object |
|
required: [containers] |
|
description: >- |
|
Pod spec for this member's engine pods. |
|
properties: |
|
containers: |
|
type: array |
|
minItems: 1 |
|
maxItems: 1 |
Expected behavior
Each member's template.metadata.labels and template.metadata.annotations should be propagated to that member's generated pod template:
- Standalone metadata to the Deployment pod template
- Leader metadata to the LWS leader template
- Worker metadata to the LWS worker template
Modelplane-managed labels required for workload selection and serving must remain intact. Collisions with reserved labels should either be rejected by validation or have clearly documented precedence.
Acceptance criteria
- Labels and annotations propagate for Standalone, Leader, and Worker members.
- Leader and Worker metadata remain independent.
- Modelplane-managed serving/workload labels cannot be accidentally overridden.
- Composition tests cover both the native and llm-d backends.
Summary
ModelDeploymentaccepts member-specific pod metadata at:These fields round-trip into the resolved
ModelReplica, but the workload composers do not merge them into the generated pod templates. As a result, the labels and annotations are silently ignored.Current behavior
modelplane/functions/compose-model-replica/function/backends/llmd.py
Lines 133 to 146 in d3609d0
modelplane/functions/compose-model-replica/function/backends/native.py
Lines 96 to 103 in d3609d0
template.metadata.labelsandtemplate.metadata.annotations:modelplane/apis/modeldeployments/definition.yaml
Lines 390 to 421 in d3609d0
Expected behavior
Each member's
template.metadata.labelsandtemplate.metadata.annotationsshould be propagated to that member's generated pod template:Modelplane-managed labels required for workload selection and serving must remain intact. Collisions with reserved labels should either be rejected by validation or have clearly documented precedence.
Acceptance criteria