Skip to content

feat: implement status reporting on ExporterSet resources#904

Merged
mangelajo merged 1 commit into
jumpstarter-dev:mainfrom
bennyz:feat/exporterset-status
Jul 22, 2026
Merged

feat: implement status reporting on ExporterSet resources#904
mangelajo merged 1 commit into
jumpstarter-dev:mainfrom
bennyz:feat/exporterset-status

Conversation

@bennyz

@bennyz bennyz commented Jul 21, 2026

Copy link
Copy Markdown
Member

Add deployment-style status conditions and replica counters to ExporterSet CRs, providing visibility into pod health, exporter availability, and scaling state on every reconcile loop.

depends on #864

tested with:

  go build -o bin/exporter-set-controller cmd/exporter-set-controller/main.go
 ./bin/exporter-set-controller --provisioner=qemu.jumpstarter.dev --health-probe-bind-address=:8082

kubectl apply -f - <<'EOF'
  apiVersion: virtualtarget.jumpstarter.dev/v1alpha1
  kind: VirtualTargetClass
  metadata:
    name: qemu-test
    namespace: jumpstarter
  spec:
    provisioner: qemu.jumpstarter.dev
    bindingMode: Immediate
    reclaimPolicy: Delete
  ---
  apiVersion: virtualtarget.jumpstarter.dev/v1alpha1
  kind: ExporterSet
  metadata:
    name: test-set
    namespace: jumpstarter
  spec:
    minReplicas: 2
    maxReplicas: 4
    minAvailableReplicas: 1
    virtualTargetClassName: qemu-test
    selector:
      matchLabels:
        exporterset: test-set
    template:
      metadata:
        labels:
          exporterset: test-set
  EOF

kubectl apply -f - <<EOF
  apiVersion: jumpstarter.dev/v1alpha1
  kind: Exporter
  metadata:
    name: exp-1
    namespace: jumpstarter
    labels:
      exporterset: test-set
    ownerReferences:
    - apiVersion: virtualtarget.jumpstarter.dev/v1alpha1
      kind: ExporterSet
      name: test-set
      uid: "$ES_UID"
      controller: true
      blockOwnerDeletion: true
  spec:
    enabled: true
  ---
  apiVersion: jumpstarter.dev/v1alpha1
  kind: Exporter
  metadata:
    name: exp-2
    namespace: jumpstarter
    labels:
      exporterset: test-set
    ownerReferences:
    - apiVersion: virtualtarget.jumpstarter.dev/v1alpha1
      kind: ExporterSet
      name: test-set
      uid: "$ES_UID"
      controller: true
      blockOwnerDeletion: true
  spec:
    enabled: true
  ---
  apiVersion: jumpstarter.dev/v1alpha1
  kind: Exporter
  metadata:
    name: exp-3
    namespace: jumpstarter
    labels:
      exporterset: test-set
    ownerReferences:
    - apiVersion: virtualtarget.jumpstarter.dev/v1alpha1
      kind: ExporterSet
      name: test-set
      uid: "$ES_UID"
      controller: true
      blockOwnerDeletion: true
  spec:
    enabled: true
  ---
  apiVersion: jumpstarter.dev/v1alpha1
  kind: Exporter
  metadata:
    name: exp-4
    namespace: jumpstarter
    labels:
      exporterset: test-set
    ownerReferences:
    - apiVersion: virtualtarget.jumpstarter.dev/v1alpha1
      kind: ExporterSet
      name: test-set
      uid: "$ES_UID"
      controller: true
      blockOwnerDeletion: true
  spec:
    enabled: false
  EOF

  # exp-1: online, idle
  kubectl patch exporter exp-1 -n jumpstarter --type=merge --subresource=status -p '{
    "status": {
      "conditions": [{"type":"Online","status":"True","lastTransitionTime":"2026-07-21T13:00:00Z","reason":"Connected","message":"connected"}]
    }
  }'

  # exp-2: online, leased
  kubectl patch exporter exp-2 -n jumpstarter --type=merge --subresource=status -p '{
    "status": {
      "conditions": [{"type":"Online","status":"True","lastTransitionTime":"2026-07-21T13:00:00Z","reason":"Connected","message":"connected"}],
      "leaseRef": {"name": "test-lease"}
    }
  }'

  # exp-3: offline
  kubectl patch exporter exp-3 -n jumpstarter --type=merge --subresource=status -p '{
    "status": {
      "conditions": [{"type":"Online","status":"False","lastTransitionTime":"2026-07-21T13:00:00Z","reason":"Disconnected","message":"offline"}]
    }
  }'

  # exp-4: online but disabled
  kubectl patch exporter exp-4 -n jumpstarter --type=merge --subresource=status -p '{
    "status": {
      "conditions": [{"type":"Online","status":"True","lastTransitionTime":"2026-07-21T13:00:00Z","reason":"Connected","message":"connected"}]
    }
  }'

kubectl get exporterset -n jumpstarter -o wide
NAME       REPLICAS   READY   AVAILABLE   UNAVAILABLE   LEASED   CLASS
test-set   4          3       1           1             1        qemu-test

kubectl get events -n jumpstarter --field-selector involvedObject.name=test-set
LAST SEEN   TYPE      REASON                 OBJECT                 MESSAGE
18m         Warning   ExporterSetDegraded    exporterset/test-set   ExporterSet test-set is degraded: 0 failed pods, 1 unavailable replicas
18m         Warning   ScalingLimited         exporterset/test-set   ExporterSet test-set scaling limited at 4 replicas
18m         Normal    ScalingUnlimited       exporterset/test-set   ExporterSet test-set scaling no longer limited
18m         Normal    ExporterSetAvailable   exporterset/test-set   ExporterSet test-set is available: 2 ready replicas
18m         Normal    ExporterSetRecovered   exporterset/test-set   ExporterSet test-set recovered: all replicas healthy
18m         Warning   ExporterSetDegraded    exporterset/test-set   ExporterSet test-set is degraded: 1 failed pods, 1 unavailable replicas

@coderabbitai

coderabbitai Bot commented Jul 21, 2026

Copy link
Copy Markdown
Contributor
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the main change: ExporterSet status reporting.
Description check ✅ Passed The description matches the changeset by describing status conditions, replica counters, and reconcile-loop updates.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
controller/internal/exporterset/reconciler.go (1)

475-481: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Replace the deprecated requeue path
Result.Requeue is deprecated in controller-runtime v0.21. Return the conflict error here instead so the standard rate-limited retry path applies.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@controller/internal/exporterset/reconciler.go` around lines 475 - 481, Update
requeueConflict to return the original conflict error instead of
ctrl.Result{Requeue: true}, while preserving the conflict log message and
standard result shape so controller-runtime’s rate-limited retry path handles
it.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@controller/internal/exporterset/reconciler.go`:
- Around line 153-158: Update the status selector assignment in the reconciler
after metav1.LabelSelectorAsSelector succeeds: use the parsed selector’s
String() value for es.Status.Selector instead of
metav1.FormatLabelSelector(&es.Spec.Selector), preserving the existing
validation and error handling.

---

Nitpick comments:
In `@controller/internal/exporterset/reconciler.go`:
- Around line 475-481: Update requeueConflict to return the original conflict
error instead of ctrl.Result{Requeue: true}, while preserving the conflict log
message and standard result shape so controller-runtime’s rate-limited retry
path handles it.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: af5dc1f8-1586-490b-a0d2-1856ac710cc2

📥 Commits

Reviewing files that changed from the base of the PR and between 3aaaa70 and aeee053.

📒 Files selected for processing (4)
  • controller/api/virtualtarget/v1alpha1/exporterset_types.go
  • controller/deploy/operator/config/crd/bases/virtualtarget.jumpstarter.dev_exportersets.yaml
  • controller/internal/exporterset/reconciler.go
  • controller/internal/exporterset/reconciler_test.go

Comment thread controller/internal/exporterset/reconciler.go Outdated
@bennyz
bennyz force-pushed the feat/exporterset-status branch from aeee053 to 0f2ae54 Compare July 21, 2026 14:50
Add deployment-style status conditions and replica counters to ExporterSet
CRs, providing visibility into pod health, exporter availability, and
scaling state on every reconcile loop.

Status fields:
 - Replica counters: total, ready, available, unavailable, leased
 - Pod phase aggregation: pending, running, failed, unknown
 - Exporter state breakdown: active, idle, disabled
 - Selector field for HPA/KEDA scale subresource compatibility

Conditions (Available, Progressing, Degraded, ScalingLimited):
 - Available: true when readyReplicas >= minReplicas
 - Progressing: true when pods pending or replicas not ready
 - Degraded: true when pods failed or exporters offline with no pending pods
 - ScalingLimited: true when at maxReplicas and need more available

Events emitted on condition transitions (e.g. ExporterSetAvailable,
ExporterSetDegraded, ScalingLimited).

Ref: JEP-0014

Signed-off-by: Benny Zlotnik <bzlotnik@redhat.com>
Assisted-by: claude-opus-4.6
@bennyz
bennyz force-pushed the feat/exporterset-status branch from 0f2ae54 to 65de675 Compare July 21, 2026 14:57
@bennyz
bennyz marked this pull request as ready for review July 21, 2026 15:35
@bennyz
bennyz requested review from kirkbrauer and mangelajo July 21, 2026 15:35

@kirkbrauer kirkbrauer left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this looks good, WDYT @mangelajo ?

Comment on lines +195 to +226
var replicas, ready, available, leased int32
var active, idle, disabled, offline int32

for i := range ownedExporters {
exp := &ownedExporters[i]
replicas++

isOnline := meta.IsStatusConditionTrue(
exp.Status.Conditions,
string(jumpstarterdevv1alpha1.ExporterConditionTypeOnline),
)
isEnabled := exp.IsEnabled()
isLeased := exp.Status.LeaseRef != nil

if isOnline {
ready++
}
if isLeased {
leased++
}
if isOnline && isEnabled && !isLeased {
available++
}

if !isEnabled {
disabled++
} else if isLeased && isOnline {
active++
} else if isOnline {
idle++
} else {
offline++

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looking at the way we calculate the different fields, it looks like available and idle will always hold the same value that represents exporters that are online, enabled, and not leased. given that, do we actually need both fields in the status? AvailableReplicas and ExportersIdle.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have thought about it and ultimately decided to keep it to allow repurposing, but I don't mind either way

@mangelajo
mangelajo added this pull request to the merge queue Jul 22, 2026
Merged via the queue into jumpstarter-dev:main with commit bbf8004 Jul 22, 2026
24 checks passed
mickume pushed a commit to mickume/jumpstarter that referenced this pull request Jul 24, 2026
…starter-dev#908)

This PR extends the Jumpstarter operator to deploy and manage
**ExporterSet provisioner controllers** as Kubernetes Deployments, as
defined in
[JEP-0014](https://jumpstarter.dev/main/contributing/jeps/JEP-0014-virtual-scalable-exporters.html).
It builds on the ExporterSet controller bootstrap in jumpstarter-dev#864.

When `spec.exporterSets.provisioners` is configured, the operator
creates one Deployment per enabled provisioner, running the shared
`jumpstarter-exporterset-controller` image with `--provisioner=<name>`.
Each provisioner gets its own ServiceAccount, Role, and RoleBinding.

### API changes

Adds `spec.exporterSets` to the `Jumpstarter` CR:

- **Global settings**: `image`, `imagePullPolicy`, `resources`
- **Per-provisioner config** (`provisioners[]`): `name`, `enabled`,
optional `image` override, `replicas`, optional `resources` override
- **`ProvisionerConfig.name` validation**: DNS-subdomain-safe pattern
(`^[a-z0-9]([a-z0-9.-]*[a-z0-9])?$`); names that collide after
sanitization are rejected at reconcile time

Adds status condition `ExporterSetControllersReady`, included in overall
`Ready` only when at least one provisioner is enabled.

### Operator behavior

- Reconciles Deployment + RBAC (SA, Role, RoleBinding) for each enabled
provisioner
- Cleans up Deployments and owned Role/RoleBinding when a provisioner is
disabled or removed (ServiceAccounts are preserved, matching existing
operator SA patterns)
- Reports `ExporterSetControllersReady` based on each enabled
provisioner Deployment's `Available` condition
- Extends operator ClusterRole with `roles` delete/patch, `pods`
get/list/watch, and `virtualtarget.jumpstarter.dev` read/status
permissions (required to create provisioner Roles with those rules)
- Default provisioner controller resources when unset: 100m CPU / 256Mi
memory requests, 500m CPU / 512Mi memory limits


### Exporter-set-controller runtime fix

The provisioner controller binary is updated to work with namespaced
Roles:

- Requires `NAMESPACE` env var (set via downward API in the Deployment)
- Restricts the controller-runtime cache to that namespace
- Sets `LeaderElectionNamespace` to the pod namespace

Without this, informers issue cluster-scoped List/Watch calls that are
rejected by the API server.

### Dev / deploy tooling

- `make deploy` and `make deploy-operator` now also build
`jumpstarter-exporterset-controller`
- `hack/deploy_with_operator.sh` deploys `qemu.jumpstarter.dev` as a
default enabled provisioner in the generated Jumpstarter CR

### Example CR snippet

```yaml
spec:
  exporterSets:
    image: quay.io/jumpstarter-dev/jumpstarter-exporterset-controller:latest
    imagePullPolicy: IfNotPresent
    resources:
      requests:
        cpu: 100m
        memory: 256Mi
    provisioners:
      - name: qemu.jumpstarter.dev
        enabled: true
        replicas: 1
        # optional per-provisioner overrides:
        # image: ...
        # resources: ...
```


## Test plan
- [x] Unit tests for resource constructors, RBAC rules, resource
defaults, and provisioner name sanitization (`exporterset_test.go`)
- [x] Controller integration tests for full lifecycle: create,
per-provisioner image/resources override, disable/cleanup, multiple
provisioners, mixed enabled/disabled, spec removal, name collision
rejection, and `ExporterSetControllersReady` status
(`jumpstarter_controller_test.go`)
- [x] `make -C controller/deploy/operator test`
- [x] Manual: `make deploy` and verify `qemu.jumpstarter.dev`
provisioner Deployment, SA, and Role come up in `jumpstarter-lab`
## Related
- [JEP-0014: Virtual Scalable
Exporters](https://jumpstarter.dev/main/contributing/jeps/JEP-0014-virtual-scalable-exporters.html)
- jumpstarter-dev#864 — ExporterSet controller bootstrap (Phase 2)
- jumpstarter-dev#904 — ExporterSet status reporting

---------

Co-authored-by: Miguel Angel Ajo Pelayo <miguelangel@ajo.es>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants