Expose Crossplane managed-resource sync state as metrics Coroot can query

[devantler]

🤖 Generated by the Agentic Engineer

Evidence: 17 of 20 Repository managed resources sat Synced: False for weeks with no signal
(parent #2820). Today kubectl get managed -A spans 109 resources across 7 API groups, of which
7 are at reason=ReconcileError and 9 unifi resources are at reason=ReconcilePaused by design.
None of that state is queryable as a metric: the cluster has no ServiceMonitor/PodMonitor CRDs
(prometheus-operator went with the kube-prometheus-stack), and Coroot's bundled cluster-agent
exposes only image under clusterAgent.kubeStateMetrics — no CustomResourceState config — so
nothing currently emits Crossplane condition state.

Audience/problem: nothing can observe, chart, or alert on a managed resource that stops
reconciling, because the underlying signal does not exist in the metrics store. This blocks every
option weighed on the parent, not just one of them.

Expected behaviour: each Crossplane managed resource's Synced condition is exposed as a
Prometheus metric in Coroot's bundled Prometheus, carrying at minimum the resource's
kind/name/namespace and its condition reason — the reason label is what makes the deliberate
ReconcilePaused state separable from a genuine ReconcileError, and the parent's measurements show
a filter without it selects exactly the wrong set.

Proposed: upstream kube-state-metrics with a CustomResourceState ConfigMap, scraped by the
cluster-agent via the pod annotations it supports (coroot.com/scrape-metrics: "true" +
coroot.com/metrics-port; nothing in k8s/ uses these yet). A pinned upstream image keeps this
declarative and Renovate-trackable and avoids standing up an image build — platform's Go module is
yaml.v3 only, so a bespoke exporter would be a project rather than an increment. Bind its
ServiceAccount to the existing aggregated crossplane-view ClusterRole, verified on the parent to
grant no core secrets; authoring a new ClusterRole is unnecessary and would trip the EKS
authorization-surface fingerprint gate.

Known cost to weigh during implementation: CustomResourceState is configured per-GVK, so covering
7 API groups means 7 entries, and a newly-installed provider needs one added. Note that trade-off in
the PR against the alternative of a small first-party exporter.

Acceptance criteria:

• Synced condition state for Crossplane managed resources is present in Coroot's bundled
  Prometheus and queryable in the UI.
• The metric carries the condition reason, and a query excluding ReconcilePaused returns
  the genuinely-failing set and not the paused one.
• up{job="crossplane-sync-exporter"} is delivered too, so "no condition series" is not
  ambiguous between a healthy fleet and a dead exporter.
• RED-proved against live data: the Repository resources at reason=ReconcileError appear.
• No new ClusterRole — amended 2026-08-10; decided, not pending. What feat(coroot): export Crossplane managed-resource conditions as metrics #2994 shipped is a
  purpose-built crossplane-sync-exporter ClusterRole granting get/list/watch on exactly
  repo.github.m.upbound.io/repositories. The alternative this criterion originally asked for —
  binding the aggregated crossplane-view — carries 49 rules, so the deviation is a
  narrowing, not a widening. The 🔐 Validate EKS Authorization gate that the criterion
  predicted would trip passed on the merged head. Do not "fix" this by rebinding to
  crossplane-view: that would widen read access from one kind to the whole aggregate.
• No access to core secrets — held; the role above grants none.
• Covers managed resources generically across API groups, not only Repository.
  Scope amended 2026-08-10 → Extend the Crossplane sync-state exporter past Repository — 90 of 110 managed resources are unobserved #3052. CustomResourceStateMetrics takes an exact GVK with no
  wildcard, so generic coverage is a separate piece of work: 110 managed resources across 7
  groups and 14 kinds exist, of which this exporter observes 20.

Two criteria above are decided rather than outstanding, and are ticked for that reason. The
record and the artifact disagreed until this amendment, and on the ClusterRole item that
disagreement pointed at the less safe option — see the 2026-08-06 comment below for the full
measurement.

Rough size: M. Delivers the signal only — alerting and Slack routing are #2987.

Part of #2820

[coderabbitai]

⚠️ Possible Duplicate Issue(s)

• Export Crossplane managed-resource conditions as metrics (first increment of #2820) #2993

🔗 Related PRs

#2445 - feat(kubescape): route runtime-detection alerts to Headlamp, Slack, and Coroot [merged]
#2459 - feat: deploy Kyverno Policy Reporter (UI + kyverno plugin) [merged]
#2794 - fix(opencost): restore usage metrics [merged]
#2994 - feat(coroot): export Crossplane managed-resource conditions as metrics [merged]
#3017 - fix(security): disposition the six unreachable-probe checkov findings [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!