Skip to content

Member pod template labels and annotations are ignored #378

Description

@sttts

Summary

ModelDeployment accepts member-specific pod metadata at:

spec:
  template:
    spec:
      engines:
      - name: example
        members:
        - role: Leader
          template:
            metadata:
              labels:
                example.com/role: leader
              annotations:
                example.com/config: leader
            spec:
              containers:
              - name: engine
                image: example/engine:latest
        - role: Worker
          worker:
            nodes: 1
          template:
            metadata:
              labels:
                example.com/role: worker
              annotations:
                example.com/config: worker
            spec:
              containers:
              - name: engine
                image: example/engine:latest

These fields round-trip into the resolved ModelReplica, but the workload composers do not merge them into the generated pod templates. As a result, the labels and annotations are silently ignored.

Current behavior

  • The llm-d backend constructs leader metadata only from Modelplane-managed labels and constructs the worker pod template without metadata:
    # Only the leader serves the OpenAI API → it carries the serving label
    # the replica's shared Service selects on, plus the role label, the
    # serving port, and the readiness probe.
    leader_pod = {
    "metadata": {"labels": {base.LABEL_SERVING: serving_label, _LABEL_ROLE: "leader"}},
    "spec": pod_spec(leader, container(leader, serving=True)),
    }
    # The worker followers don't serve the OpenAI API, so they carry no
    # serving label - the replica's Service must never route to them. LWS
    # manages their gang membership labels itself.
    worker_pod = {
    "spec": pod_spec(worker, container(worker, serving=False)),
    }
  • The native backend likewise constructs pod metadata only from Modelplane-managed labels:
    if tmpl.spec.imagePullSecrets:
    pod_spec["imagePullSecrets"] = [s.model_dump(exclude_none=True) for s in tmpl.spec.imagePullSecrets]
    deployment = {
    "apiVersion": "apps/v1",
    "kind": "Deployment",
    "metadata": {"name": name, "namespace": base.REMOTE_NAMESPACE},
    "spec": {
  • The API explicitly exposes template.metadata.labels and template.metadata.annotations:
    template:
    type: object
    description: >-
    Pod template for this member's engine pods. A
    curated subset of PodTemplateSpec.
    properties:
    metadata:
    type: object
    description: >-
    Metadata applied to the member's pods. Useful
    for labels and annotations that control
    cluster-level features like service mesh
    injection.
    properties:
    labels:
    type: object
    additionalProperties:
    type: string
    annotations:
    type: object
    additionalProperties:
    type: string
    spec:
    type: object
    required: [containers]
    description: >-
    Pod spec for this member's engine pods.
    properties:
    containers:
    type: array
    minItems: 1
    maxItems: 1

Expected behavior

Each member's template.metadata.labels and template.metadata.annotations should be propagated to that member's generated pod template:

  • Standalone metadata to the Deployment pod template
  • Leader metadata to the LWS leader template
  • Worker metadata to the LWS worker template

Modelplane-managed labels required for workload selection and serving must remain intact. Collisions with reserved labels should either be rejected by validation or have clearly documented precedence.

Acceptance criteria

  • Labels and annotations propagate for Standalone, Leader, and Worker members.
  • Leader and Worker metadata remain independent.
  • Modelplane-managed serving/workload labels cannot be accidentally overridden.
  • Composition tests cover both the native and llm-d backends.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status
    Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions