feat(coroot): export Crossplane managed-resource conditions as metrics - #2994
Conversation
A Crossplane managed resource can report Ready=True while Synced=False. Every health check keyed on Ready then reads the fleet as healthy while no declared setting is reaching the provider. Measured on the live cluster, 3 of 20 Repository resources are in exactly that state right now. Nothing in the cluster observes the Synced condition, and Coroot's bundled Prometheus exposes no scrape configuration, so the series has to be pushed in via remote-write. This mirrors the opencost usage-scraper: kube-state-metrics renders the conditions as custom resource state, a co-located Prometheus agent scrapes it over loopback and remote-writes to coroot-prometheus. Ready is exported alongside Synced so the misleading pair is expressible as a single query. Ships default-off; activation and the alerting rule are separate, independently reversible changes.
Every workload in this repo that automounts the token carries a CKV_K8S_38 justification; this one did not.
✅
|
CI is settled at @coderabbitai review Worth a look at two things in particular:
|
🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '--- PR metadata and changed files ---'
gh pr view 2994 --repo devantler-tech/platform --json number,title,baseRefName,headRefName,files --jq '
"PR #\(.number): \(.title)\nbase=\(.baseRefName) head=\(.headRefName)",
(.files[] | "\(.path)")
printf '%s\n' '--- Diff ---'
gh pr diff 2994 --repo devantler-tech/platform --color=never
printf '%s\n' '--- Candidate manifest files ---'
fd -t f -E .git -E node_modules 'config-map\.yaml|configmap\.yaml|.*usage.*scraper.*|.*coroot.*' . | sort
printf '%s\n' '--- Kubernetes resource references ---'
rg -n -i -C 4 'CustomResourceStateMetrics|kube-state-metrics|usage-scraper|127\.0\.0\.1|localhost|metrics-port|--web\.listen-address|scrape' . \
-g '*.yaml' -g '*.yml' -g '*.json' -g '!**/node_modules/**' || trueLength of output: 1213 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '--- PR metadata ---'
gh pr view 2994 --repo devantler-tech/platform \
--json number,title,baseRefName,headRefName \
--jq '. as $pr | "PR #\($pr.number): \($pr.title)\nbase=\($pr.baseRefName) head=\($pr.headRefName)"'
printf '%s\n' '--- Changed files ---'
gh pr view 2994 --repo devantler-tech/platform --json files \
--jq '.files[] | .path'
printf '%s\n' '--- Diff ---'
gh pr diff 2994 --repo devantler-tech/platform --color=never
printf '%s\n' '--- Candidate manifest files ---'
fd -t f -E .git -E node_modules -i 'config-map\.yaml|configmap\.yaml|.*usage.*scraper.*|.*coroot.*' . | sort
printf '%s\n' '--- Kubernetes resource references ---'
rg -n -i -C 4 'CustomResourceStateMetrics|kube-state-metrics|usage-scraper|127\.0\.0\.1|localhost|metrics-port|--web\.listen-address|scrape' . \
-g '*.yaml' -g '*.yml' -g '*.json' -g '!**/node_modules/**' || trueLength of output: 90604 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '--- Repository managed resources and provider references ---'
rg -n -i -C 8 \
'apiVersion:\s*repo\.github\.m\.upbound\.io/|kind:\s*Repository\b|provider-github|github\.m\.upbound\.io' \
. -g '*.yaml' -g '*.yml' -g '!**/node_modules/**' || true
printf '%s\n' '--- CRD definitions and condition fixtures ---'
fd -t f -E .git -E node_modules -i '.*crd.*|.*repository.*|.*crossplane.*' . | sort | head -300
rg -n -i -C 8 \
'status:\s*$|conditions:\s*$|Synced|Ready|repo\.github\.m\.upbound\.io' \
. -g '*.yaml' -g '*.yml' -g '*.json' -g '!**/node_modules/**' || true
printf '%s\n' '--- Related validation and component tests ---'
rg -n -i -C 5 \
'usage-scraper|crossplane-sync-exporter|custom-resource-state|kube-state-metrics|CustomResourceStateMetrics' \
scripts k8s -g '*.sh' -g '*.yaml' -g '*.yml' -g '*.md' 2>/dev/null || trueLength of output: 201283 🤖 web_search executed:
Length of output: 15382 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '--- Exact Repository GVK declarations ---'
rg -n -i -C 6 '^apiVersion:[[:space:]]*repo\.github\.m\.upbound\.io/|^kind:[[:space:]]*Repository[[:space:]]*$' \
. -g '*.yaml' -g '*.yml' -g '!**/node_modules/**' || true
printf '%s\n' '--- Provider package and GitHub configuration sources ---'
rg -n -i -C 6 'provider-github|provider-family-github|github.*upbound|repo\.github\.m\.upbound\.io' \
k8s .github -g '*.yaml' -g '*.yml' -g '*.json' -g '*.md' 2>/dev/null || true
printf '%s\n' '--- OpenCost contract structure and CI registration ---'
sed -n '1,240p' scripts/tests/test-opencost-usage-scraper.sh
rg -n -C 4 'test-opencost-usage-scraper\.sh|Validate Manifests|scripts/tests' \
.github/workflows -g '*.yaml' -g '*.yml' 2>/dev/null || true
printf '%s\n' '--- Existing PR review comments ---'
gh pr view 2994 --repo devantler-tech/platform --commentsLength of output: 85056
The The two-container design is correct. Both containers share the Pod network namespace. kube-state-metrics binds to I found one regression risk: the PR does not add a rendered-component contract test for the exporter. The existing Please add a ✅ Action performedReview finished.
|
Verification recordStatic.
Config correctness, against live data. The failure mode that matters here is a wrong path yielding zero series silently, which is indistinguishable from a healthy cluster. So I simulated the declared
Every declared JSON path resolves against real objects, and that last row is this PR's whole reason for existing — measured, not argued. The condition-Gauge shape and the string→numeric conversion ( What is NOT claimed. I have not observed the metric arrive in The image digest was resolved from |
|
Warning Review limit reached
Next review available in: 31 minutes You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Repository YAML (base), Organization UI (inherited) Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (8)
📝 WalkthroughWalkthroughAdds a default-off Kustomize component for Crossplane sync metrics. It creates a ServiceAccount, read-only RBAC for Possibly related issues
Possibly related PRs
Important Pre-merge checks failedPlease resolve all errors before merging. Addressing warnings is optional. ❌ Failed checks (1 error)
✅ Passed checks (4 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@k8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/kustomization.yaml`:
- Around line 24-29: Add the crossplane sync exporter rendered-component
contract test as test-crossplane-sync-exporter.sh, covering both cluster
overlays, default-off behavior, and required RBAC, ConfigMap, Deployment,
metric-filter, and 127.0.0.1:8080 wiring. Register this test in the CI workflow
defined by ci.yaml so it runs before merge.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Repository YAML (base), Organization UI (inherited)
Review profile: ASSERTIVE
Plan: Pro Plus
Run ID: e598153a-6296-4d69-8ff8-bec964375301
📒 Files selected for processing (6)
k8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/cluster-role-binding.yamlk8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/cluster-role.yamlk8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/config-map.yamlk8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/deployment.yamlk8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/kustomization.yamlk8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/service-account.yaml
📜 Review details
🧰 Additional context used
📓 Path-based instructions (3)
k8s/**/*.yaml
📄 CodeRabbit inference engine (AGENTS.md)
k8s/**/*.yaml: Kubernetes base files underk8s/bases/are immutable; modify them through Kustomizepatches:in provider or cluster overlays.
Preserve Flux dependency order:bootstrap→infrastructure-controllers→infrastructure→apps.
Use one Kubernetes resource per file, except explicitly whitelisted vendored CDI and KubeVirt operator bundles.
Use kebab-case directories and Kind-led filenames in component folders; qualify filenames with a purpose when multiple resources share a Kind.
Before a manifest PR, run schema-aware KSail validation when available, otherwise build bothk8s/clusters/local/andk8s/clusters/prod/withkubectl kustomize; use client-side dry-run for changed individual manifests.
Files:
k8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/service-account.yamlk8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/cluster-role-binding.yamlk8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/cluster-role.yamlk8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/deployment.yamlk8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/config-map.yamlk8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/kustomization.yaml
**/*.{yaml,yml}
📄 CodeRabbit inference engine (AGENTS.md)
Use Conventional Commit messages and create pull requests as drafts; keep changes small and focused.
Files:
k8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/service-account.yamlk8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/cluster-role-binding.yamlk8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/cluster-role.yamlk8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/deployment.yamlk8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/config-map.yamlk8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/kustomization.yaml
k8s/bases/infrastructure/**/*.yaml
📄 CodeRabbit inference engine (AGENTS.md)
k8s/bases/infrastructure/**/*.yaml: Place infrastructure resources in component-first folders; split CRs into plural-Kind folders only for dependency-order or cluster-scoped/cross-cutting reasons.
FeatureFlag CRs belong in theinfrastructurelayer, never the controllers layer, because their CRD is installed by the controller.
Files:
k8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/service-account.yamlk8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/cluster-role-binding.yamlk8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/cluster-role.yamlk8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/deployment.yamlk8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/config-map.yamlk8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/kustomization.yaml
🧠 Learnings (1)
📚 Learning: 2026-07-01T21:13:36.950Z
Learnt from: devantler
Repo: devantler-tech/platform PR: 2359
File: k8s/bases/apps/actual-budget/helm-release.yaml:62-111
Timestamp: 2026-07-01T21:13:36.950Z
Learning: When reviewing Kustomize/Helm YAML in this repo, keep the base vs provider overlay split: `k8s/bases/apps/**` and `k8s/bases/infrastructure/**` should contain each app’s full, environment-agnostic configuration (including base-level postRenderer Kustomize patches such as deployment strategy, topology spread, probes, and env injection). `k8s/providers/{docker,hetzner}/**` should only add small provider-specific deltas (e.g., `interval`, `persistence.size`) via patch files (like `k8s/providers/<provider>/apps/<app>/patches/helm-release-patch.yaml`). If configuration is identical across providers (e.g., OIDC/OAuth env vars where `${domain}` is resolved per cluster via envsubst), it belongs in the base and must not be duplicated into provider overlays.
Applied to files:
k8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/service-account.yamlk8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/cluster-role-binding.yamlk8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/cluster-role.yamlk8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/deployment.yamlk8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/config-map.yamlk8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/kustomization.yaml
🪛 Trivy (0.72.0)
k8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/deployment.yaml
[warning] 41-74: Restrict container images to trusted registries
Container kube-state-metrics in deployment crossplane-sync-exporter (namespace: observability) uses an image from an untrusted registry.
Rule: KSV-0125
(IaC/Kubernetes)
[warning] 75-104: Restrict container images to trusted registries
Container prometheus-agent in deployment crossplane-sync-exporter (namespace: observability) uses an image from an untrusted registry.
Rule: KSV-0125
(IaC/Kubernetes)
🔇 Additional comments (5)
k8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/service-account.yaml (1)
1-6: LGTM!k8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/cluster-role.yaml (1)
1-17: LGTM!k8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/cluster-role-binding.yaml (1)
1-12: LGTM!k8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/config-map.yaml (1)
1-68: LGTM!k8s/bases/infrastructure/coroot/components/crossplane-sync-exporter/deployment.yaml (1)
1-111: LGTM!
A wrong GVK, a renamed series, or a scrape target drifting from the kube-state-metrics listen address all export zero series, and an empty result is indistinguishable from a fleet with nothing wrong. Schema validation cannot see any of it, so assert the rendered contract instead. Follows the test-opencost-usage-scraper.sh contract for the sibling component.
…tract A substring assertion passes on a longer value, so renaming the metric prefix from crossplane to crossplaneX satisfied 'metricNamePrefix: crossplane' and the ablation did not fire. Compare the trimmed line exactly instead.
@coderabbitai review New head CI at this head: 20 pass / 0 fail. I confirmed the new step The correction is the part worth a second look: the test's first version used a substring match, so renaming |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '--- PR changed files and current diff ---'
gh pr view 2994 --repo devantler-tech/platform --json files --jq '.files[] | .path'
gh pr diff 2994 --repo devantler-tech/platform --color=never -- \
scripts/tests/test-crossplane-sync-exporter.sh \
.github/workflows/ci.yaml \
k8s/bases/infrastructure/coroot/components/crossplane-sync-exporter
printf '%s\n' '--- Candidate file locations ---'
fd -t f -E .git -E node_modules \
'test-crossplane-sync-exporter\.sh|ci\.ya?ml|config-map\.yaml|deployment\.yaml|kustomization\.yaml' . | sort
printf '%s\n' '--- Shell test structure ---'
test_file="$(fd -t f -E .git -E node_modules '^test-crossplane-sync-exporter\.sh$' . | head -n 1)"
if [ -n "$test_file" ]; then
ast-grep outline "$test_file" --items all || true
cat -n "$test_file"
fi
printf '%s\n' '--- CI registration context ---'
rg -n -C 10 'test-crossplane-sync-exporter\.sh|Validate Crossplane sync exporter metrics path' \
.github/workflows -g 'ci.yaml' -g 'ci.yml' || true
printf '%s\n' '--- Exporter manifest structure ---'
component_dir="$(fd -t d -E .git -E node_modules '^crossplane-sync-exporter$' k8s | head -n 1)"
if [ -n "$component_dir" ]; then
for file in "$component_dir"/config-map.yaml "$component_dir"/deployment.yaml "$component_dir"/kustomization.yaml; do
if [ -f "$file" ]; then
printf '\n--- %s ---\n' "$file"
cat -n "$file"
fi
done
fiLength of output: 2424 🏁 Script executed: #!/bin/bash
set -euo pipefail
printf '%s\n' '--- Relevant PR diff ---'
gh pr diff 2994 --repo devantler-tech/platform --color=never
printf '%s\n' '--- Contract test ---'
test_file="$(fd -t f -E .git -E node_modules '^test-crossplane-sync-exporter\.sh$' . | head -n 1 || true)"
if [ -z "$test_file" ]; then
echo 'Contract test was not found.'
else
cat -n "$test_file"
fi
printf '%s\n' '--- CI registration ---'
ci_file="$(fd -t f -E .git -E node_modules '^ci\.ya?ml$' .github/workflows | head -n 1 || true)"
if [ -z "$ci_file" ]; then
echo 'CI workflow was not found.'
else
rg -n -C 12 'test-crossplane-sync-exporter\.sh|Validate Crossplane sync exporter metrics path' "$ci_file" || true
fi
printf '%s\n' '--- Exporter manifests checked by the contract ---'
component_dir="$(fd -t d -E .git -E node_modules '^crossplane-sync-exporter$' k8s | head -n 1 || true)"
if [ -z "$component_dir" ]; then
echo 'Exporter component directory was not found.'
else
for file in "$component_dir"/config-map.yaml "$component_dir"/deployment.yaml "$component_dir"/kustomization.yaml; do
if [ -f "$file" ]; then
printf '\n--- %s ---\n' "$file"
cat -n "$file"
fi
done
fiLength of output: 50409
The contract now protects the required GVK, condition paths, metric-name coupling, relabel filter, ConfigMap, Deployment, default-off state, and The workflow both detects changes to I found no follow-up issues in this focused review. 🐇 ✅ If you found this review helpful, would you consider giving us a shout-out on X? Thank you for using CodeRabbit! You are interacting with an AI system. |
@coderabbitai full review |
|
|
@codex review |
CodeRabbit refused at this head — included-review limit reached under its fair-usage policy, next included review in 31 minutes. That is a service state, not a verdict on the change, so it does not gate the merge; recording it and advancing one lane rather than waiting. Note for the record: its commit status still reads @codex review Focus, if useful: the |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
1 similar comment
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Codex refused at this head too — account code-review usage limits reached, no retry window stated (06:03:08Z, direct on this PR). Advancing to the third and last lane. The next comment is a bare trigger with nothing else in it: Bugbot exact-matches the whole comment body, so any extra line silently voids the request, and a voided request is indistinguishable from a dead lane. |
|
@cursor review |
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_83a64a1b-fb14-4885-8092-bc556482e7d5) |
Metric relabeling excludes automatically generated timeseries, so the exporter's liveness signal survives the filter. Without it, an absent condition series would be ambiguous between a healthy fleet and a dead exporter — the same silent-sensor failure this component exists to surface.
devantler
left a comment
devantler
left a comment
There was a problem hiding this comment.
🤖 Generated by the Agentic Engineer
Self-review (fallback — CodeRabbit, Codex and Cursor Bugbot unavailable)
Reviewed commit: 1b748f28ce511b3a06449327b6544382793c4a88
All three external lanes were tried in priority order at this exact head, each with direct per-PR
evidence, and none will deliver a review:
- CodeRabbit —
state=success :: description=Review rate limitedat this head (06:02Z), summary
comment carries quota markers, zero review objects. An earlier request at 05:48Z was additionally
swallowed as a thread reply (review_comment_addressed); a barefull reviewre-trigger was
accepted at 06:01Z and then hit the limit. No retry window stated. - Codex —
You have reached your Codex usage limits for code reviews(06:03:08Z, 28s after request). - Cursor Bugbot — check-run
neutralwithoutput.title: Error, plus acursor[bot]comment
Bugbot couldn't run - usage limit reached(06:04:3xZ). Zero inline comments — the never-ran shape.
What I verified against the live cluster (not from the diff alone)
- GVK is exact and correct.
repositories.repo.github.m.upbound.ioserves onlyv1alpha1
(served+storage), live objects reportrepo.github.m.upbound.io/v1alpha1, scope Namespaced.
CustomResourceStateMetricstakes no wildcard, so a drifted version would export zero series; it
does not drift. The config'snamespace: [metadata, namespace]label is right for a namespaced kind. - Condition value domain is exactly
{True, False}across all live conditions, which is what the
Gauge's string→numeric conversion handles. No third value to mis-map. - The fixture is real and non-zero right now: 20
RepositoryMRs — 17Synced=True, 3
Synced=False, and 3 withReady=TruewhileSynced=False— the exact defect this metric
exists to surface. - Metric-name coupling holds:
metricNamePrefix: crossplane+name: managed_resource_condition
→crossplane_managed_resource_condition, which is precisely thekeepregex in the scrape config. - RBAC is minimal — get/list/watch on one kind, no wildcard; the test asserts the absence of one.
Checked and cleared (recorded so it is not re-litigated)
I suspected the metric_relabel_configs action: keep would also drop up{job=...}, which would
reintroduce this component's own failure mode — an exporter that dies looking identical to a healthy
fleet. It does not. Prometheus generates up and the scrape_* series in the scrape loop rather
than through metric relabeling; scrape_samples_post_metric_relabeling exists precisely to report the
post-relabel count, so those synthetic series are produced around that step. The deployed
opencost/usage-scraper uses the identical pattern. Worth re-confirming empirically at activation
time, since the alerting rule in #2987 will depend on up being present to distinguish
"exporter down" from "nothing wrong".
Scope note
Coverage is deliberately partial — Repository only, of 16 managed-resource CRDs — because
CustomResourceStateMetrics requires an exact GVK per kind and Repository is the kind carrying both
a positive and a negative fixture. The component ships default-off (unreferenced), so CI's overlay
builds do not exercise it; the standalone components:-referencing contract test is the only thing
that does, and that is why it exists.
Verdict: no P0/P1 findings.
Review provenance note (honesty about what was read at which commit). My self-review posted against The delta is +9 lines, one file, comment-only: a YAML comment inside the Worth noting: that commit and my "Checked and cleared" section were produced independently and reached CI is re-running at |
devantler
left a comment
devantler
left a comment
There was a problem hiding this comment.
🤖 Generated by the Agentic Engineer
Self-review (fallback — CodeRabbit, Codex and Cursor Bugbot unavailable)
Reviewed commit: 1b748f28ce511b3a06449327b6544382793c4a88
All three lanes were tried at this head, in priority order, one at a time, with direct per-PR evidence — not inherited from another PR's digest:
| Lane | Outcome | Evidence |
|---|---|---|
| CodeRabbit | refused | Review rate limited — included-review limit under Fair Usage; next included review in 31 minutes (06:01Z) |
| Codex | refused | You have reached your Codex usage limits for code reviews, no retry window (06:03:08Z) |
| Cursor Bugbot | never ran | check-run conclusion=neutral / output.title=Error, Bugbot run failed after 1 second, 0 inline comments; cursor[bot]: usage limit reached (06:04:31Z) |
A provider's billing state is not a verdict on this change, so it does not gate the merge. Two of the three refusals need a human to lift a spend limit; neither is retryable on a timer.
What I checked, and how
The failure mode that matters here is silent. A wrong path, a drifted GVK or a renamed metric exports zero series, and zero series is indistinguishable from a fleet with nothing wrong. So every check below was run against real data or real documentation rather than reasoned:
- Config paths, against the live cluster. Simulated the declared extraction over the 20 real
Repositoryresources: 60 series, conditionsSynced/Ready/LastAsyncOperation, value domain exactlyTrue/False, 0 nullreasonlabels. Every declared JSON path resolves.Synced=Falseon 3,Synced=Trueon 17, and 3 withReady=TruewhileSynced=False— this PR's whole reason for existing, measured and non-zero. - kube-state-metrics flags.
--custom-resource-state-onlytogether with--custom-resource-state-config-fileis the documented exclusive-custom-resource mode;--host/--port/--telemetry-host/--telemetry-portall exist. Checked against the upstream CLI reference, since a wrong flag crashloops the container and I had asserted these in a test without verifying them. upis not dropped by the keep-filter. Metric relabeling excludes automatically generated timeseries, soup{job="crossplane-sync-exporter"}still reaches the store. This one nearly went the other way: I had it written up as a defect before checking. It matters because withoutup, an absent condition series is ambiguous between healthy fleet and dead exporter — the same silent-sensor failure this component exists to surface. Recorded in the config, because #2987's alert depends on it.- Embedded config parses. Both block scalars are valid YAML and structurally correct (
kind: CustomResourceStateMetrics, 1 resource, GVKrepo.github.m.upbound.io/v1alpha1/Repository, prefixcrossplane, metricmanaged_resource_condition,Gauge). Schema validation treats block scalars as opaque strings — this repo has a dedicated CI step for exactly that blind spot. - Deploy safety. The PR touches only the unreferenced component, its test, and one CI step. The exporter appears 0 times in the rendered prod overlay, so the
merge_groupprod deploy is a no-op for it. - The test actually tests. 8 ablations, each moving a mechanism rather than deleting it, each firing its own distinct assertion, with baseline and post-revert control both passing.
Deliberate trade-offs, stated rather than left implicit
- Both containers share the pod's service-account token, so the Prometheus agent holds a token it never uses. Splitting them would require a Service and a network-reachable metrics endpoint, which is a worse exposure for a sidecar that only scrapes loopback. The shared pod is the smaller surface.
- Coverage is one GVK, not all 16.
CustomResourceStateMetricsmatches an exactgroupVersionKind— no wildcard, nocategories: managedselector — and CI has no cluster, so nothing in the repo could detect drift in a 16-entry list. Starting with the kind that carries the observed failure proves the pipeline before committing to a list that would rot silently. Extending it is separate work with its own detection story.
Not claimed
I have not observed the metric arrive in coroot-prometheus. The component ships unreferenced and I hold read-only cluster access, so there is nothing running to observe. That check belongs to the activation change: query crossplane_managed_resource_condition and expect agreement with a live kubectl read — 3 Synced=False, 17 Synced=True. #2986 therefore goes to 📊 Verifying, not Done.
Verdict: no P0/P1 findings.
Readiness — promoting at
|
1 participant
Why
A Crossplane managed resource can sit with
Ready=TruewhileSynced=False. In thatstate nothing we declare is actually reaching the provider, but every health check
keyed on
Readyreads the fleet as healthy — so the control is voided silently whilethe config file still reads as the truth. 3 of 20 repository resources are in exactly
that state on the cluster right now. Nothing today would surface it.
What
Adds the missing metric, which is the prerequisite for alerting on it — the rule cannot
be written before the series exists. Off-the-shelf images, no bespoke code: it follows
the same shape as the existing opencost usage-scraper.
Both
SyncedandReadyare exported, so the misleading pair can be caught in onequery rather than inferred.
Ships default-off (the component is unreferenced), so turning it on is a separate,
independently reversible change — as is the alert itself and extending coverage beyond
repository resources to the other managed-resource kinds.
Fixes #2986
Part of #2820