[release-4.22] CNTRLPLANE-4321: Cherry-pick kube-scheduler metrics support by michaelryanmcneill · Pull Request #9490 · openshift/hypershift

michaelryanmcneill · 2026-09-02T17:19:04Z

What this PR does / why we need it:

Cherry-picks PRs #8489 and #8680 to release-4.22 to enable Prometheus metrics scraping for the kube-scheduler control plane component.

PR #8489 adds a CA-signed serving certificate, Service, and ServiceMonitor for the kube-scheduler, replacing the previously auto-generated self-signed certificates. This enables proper mTLS authentication for Prometheus metrics scraping of /metrics and /metrics/resources endpoints.

PR #8680 adds the necessary RBAC (ClusterRole and ClusterRoleBinding) in the hosted cluster for the system:kube-scheduler service account to access the /metrics/resources endpoint.

Conflict resolution notes:

hostedcontrolplane_controller.go: The v1 reconcileOLMAndMiscCerts function was refactored out in release-4.22. The scheduler cert reconciliation was added directly to the existing reconcilePKI function instead.
kube_scheduler/component.go: Kept util.AvailabilityProberOpts{} (release-4.22) instead of podspec.AvailabilityProberOpts{} (main), while adding the ServiceMonitor manifest adapter.

Which issue(s) this PR fixes:

Fixes CNTRLPLANE-4314

Special notes for your reviewer:

The conflict resolution in hostedcontrolplane_controller.go is the key area to review — the scheduler cert was placed between KCM and CVO certs in reconcilePKI, matching the logical placement from the original PR.
PR CNTRLPLANE-3954: add kube-scheduler metrics endpoint e2e coverage #9159 (e2e tests) was intentionally not backported because it gates on Version423 (would be skipped on 4.22) and depends on test/e2e/v2/util which doesn't exist on this branch.

Checklist:

Subject and description added to both, commit and PR.
Relevant issues have been referenced.
This change includes docs.
This change includes unit tests.

…erts The kube-scheduler previously auto-generated self-signed serving certificates via --cert-dir. This change adds a CA-signed serving certificate, a Service, and a ServiceMonitor to enable Prometheus metrics scraping with proper mTLS authentication.

Add unit tests for scheduler component options, ServiceMonitor adapter, and server certificate reconciliation. Regenerate test fixtures after adding Service, ServiceMonitor, and CA-signed serving certificate support.

The built-in system:monitoring ClusterRole only grants access to /metrics. This adds a ClusterRole and ClusterRoleBinding in the guest cluster to allow the metrics client to scrape /metrics/resources on kube-scheduler, which exposes kube_pod_resource_request and kube_pod_resource_limit metrics.

openshift-merge-bot · 2026-09-02T17:19:07Z

Pipeline controller notification
This repo is configured to use the pipeline controller. Second-stage tests will be triggered either automatically or after lgtm label is added, depending on the repository configuration. The pipeline controller will automatically detect which contexts are required and will utilize /test Prow commands to trigger the second stage.

For optional jobs, comment /test ? to see a list of all defined jobs. To trigger manually all jobs from second stage use /pipeline required command.

This repository is configured in: LGTM mode

openshift-ci-robot · 2026-09-02T17:19:09Z

@michaelryanmcneill: This pull request references CNTRLPLANE-4314 which is a valid jira issue.

Details

In response to this:

What this PR does / why we need it:

Cherry-picks PRs #8489 and #8680 to release-4.22 to enable Prometheus metrics scraping for the kube-scheduler control plane component.

PR #8489 adds a CA-signed serving certificate, Service, and ServiceMonitor for the kube-scheduler, replacing the previously auto-generated self-signed certificates. This enables proper mTLS authentication for Prometheus metrics scraping of /metrics and /metrics/resources endpoints.

PR #8680 adds the necessary RBAC (ClusterRole and ClusterRoleBinding) in the hosted cluster for the system:kube-scheduler service account to access the /metrics/resources endpoint.

Conflict resolution notes:

hostedcontrolplane_controller.go: The v1 reconcileOLMAndMiscCerts function was refactored out in release-4.22. The scheduler cert reconciliation was added directly to the existing reconcilePKI function instead.

kube_scheduler/component.go: Kept util.AvailabilityProberOpts{} (release-4.22) instead of podspec.AvailabilityProberOpts{} (main), while adding the ServiceMonitor manifest adapter.

Which issue(s) this PR fixes:

Fixes CNTRLPLANE-4314

Special notes for your reviewer:

The conflict resolution in hostedcontrolplane_controller.go is the key area to review — the scheduler cert was placed between KCM and CVO certs in reconcilePKI, matching the logical placement from the original PR.

PR CNTRLPLANE-3954: add kube-scheduler metrics endpoint e2e coverage #9159 (e2e tests) was intentionally not backported because it gates on Version423 (would be skipped on 4.22) and depends on test/e2e/v2/util which doesn't exist on this branch.

Checklist:

Subject and description added to both, commit and PR.

Relevant issues have been referenced.

This change includes docs.

This change includes unit tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

coderabbitai · 2026-09-02T17:19:13Z

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository YAML (base), Central YAML (inherited)

Review profile: CHILL

Plan: Enterprise

Run ID: 0ad0fd06-a589-4eb2-bfee-7740f05c6280

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

_{Comment @coderabbitai help to get the list of available commands.}

michaelryanmcneill · 2026-09-02T17:20:50Z

/label area/control-plane-operator
/label area/hypershift-operator

openshift-ci · 2026-09-02T17:21:02Z

@michaelryanmcneill: The label(s) /label area/control-plane-operator , /label area/hypershift-operator cannot be applied. These labels are supported: acknowledge-critical-fixes-only, platform/aws, platform/azure, platform/baremetal, platform/google, platform/libvirt, platform/openstack, ga, tide/merge-method-merge, tide/merge-method-rebase, tide/merge-method-squash, px-approved, docs-approved, qe-approved, ux-approved, no-qe, rebase/manual, cluster-config-api-changed, run-integration-tests, verified, ready-for-human-review, reliability, approved, backport-risk-assessed, bugzilla/valid-bug, cherry-pick-approved, ci/severity-critical, jira/skip-dependent-bug-check, jira/valid-bug, ok-to-test, stability-fix-approved, staff-eng-approved. Is this label configured under labels -> additional_labels or labels -> restricted_labels in plugin.yaml?

Details

In response to this:

/label area/control-plane-operator
/label area/hypershift-operator

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

The cherry-picked fixtures from main had stale desired-state-hash values that don't match the release-4.22 CPOv2 framework output. Regenerated with UPDATE=true go test. Signed-off-by: michaelryanmcneill <michael@michaelryanmcneill.com>

michaelryanmcneill · 2026-09-02T17:43:11Z

/test "Verify / Verify"

codecov · 2026-09-02T17:52:39Z

Codecov Report

❌ Patch coverage is 64.21053% with 34 lines in your changes missing coverage. Please review.
✅ Project coverage is 36.57%. Comparing base (2f5225b) to head (57bcd6d).
⚠️ Report is 9 commits behind head on release-4.22.

Files with missing lines	Patch %	Lines
support/metrics/sets.go	50.00%	14 Missing ⚠️
...igoperator/controllers/resources/manifests/rbac.go	0.00%	12 Missing ⚠️
.../hostedcontrolplane/v2/kube_scheduler/component.go	0.00%	4 Missing ⚠️
...ostedcontrolplane/hostedcontrolplane_controller.go	50.00%	2 Missing and 1 partial ⚠️
...or/controllers/hostedcontrolplane/manifests/pki.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@               Coverage Diff                @@
##           release-4.22    #9490      +/-   ##
================================================
+ Coverage         36.53%   36.57%   +0.03%     
================================================
  Files               777      779       +2     
  Lines             95363    95458      +95     
================================================
+ Hits              34840    34909      +69     
- Misses            57682    57707      +25     
- Partials           2841     2842       +1

Files with missing lines	Coverage Δ
...or/controllers/hostedcontrolplane/pki/scheduler.go	`100.00% <100.00%> (ø)`
...edcontrolplane/v2/kube_scheduler/servicemonitor.go	`100.00% <100.00%> (ø)`
...igoperator/controllers/resources/rbac/reconcile.go	`2.98% <100.00%> (+2.98%)`	⬆️
...rconfigoperator/controllers/resources/resources.go	`51.19% <100.00%> (+0.03%)`	⬆️
...or/controllers/hostedcontrolplane/manifests/pki.go	`0.00% <0.00%> (ø)`
...ostedcontrolplane/hostedcontrolplane_controller.go	`36.37% <50.00%> (+0.03%)`	⬆️
.../hostedcontrolplane/v2/kube_scheduler/component.go	`26.08% <0.00%> (+26.08%)`	⬆️
...igoperator/controllers/resources/manifests/rbac.go	`0.00% <0.00%> (ø)`
support/metrics/sets.go	`5.97% <50.00%> (+3.62%)`	⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

michaelryanmcneill · 2026-09-02T17:55:26Z

/test verify-deps

michaelryanmcneill · 2026-09-02T18:00:27Z

/test e2e-aws-upgrade-hypershift-operator
/test e2e-aws
/test e2e-v2-aws

michaelryanmcneill · 2026-09-02T21:20:57Z

/verified by e2e-aws (TestCreateCluster) — kube-scheduler ServiceMonitor with /metrics and /metrics/resources mTLS endpoints, Service on port 10259, and hypershift-metrics-resources-reader ClusterRole/ClusterRoleBinding all confirmed in must-gather artifacts

openshift-ci-robot · 2026-09-02T21:21:09Z

@michaelryanmcneill: This PR has been marked as verified by e2e-aws (TestCreateCluster) — kube-scheduler ServiceMonitor with /metrics and /metrics/resources mTLS endpoints,Service on port 10259,and hypershift-metrics-resources-reader ClusterRole/ClusterRoleBinding all confirmed in must-gather artifacts.

Details

In response to this:

/verified by e2e-aws (TestCreateCluster) — kube-scheduler ServiceMonitor with /metrics and /metrics/resources mTLS endpoints, Service on port 10259, and hypershift-metrics-resources-reader ClusterRole/ClusterRoleBinding all confirmed in must-gather artifacts

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2026-09-03T07:53:47Z

@michaelryanmcneill: This pull request references CNTRLPLANE-4321 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.22.0" version, but no target version was set.

Details

In response to this:

What this PR does / why we need it:

Cherry-picks PRs #8489 and #8680 to release-4.22 to enable Prometheus metrics scraping for the kube-scheduler control plane component.

PR #8489 adds a CA-signed serving certificate, Service, and ServiceMonitor for the kube-scheduler, replacing the previously auto-generated self-signed certificates. This enables proper mTLS authentication for Prometheus metrics scraping of /metrics and /metrics/resources endpoints.

PR #8680 adds the necessary RBAC (ClusterRole and ClusterRoleBinding) in the hosted cluster for the system:kube-scheduler service account to access the /metrics/resources endpoint.

Conflict resolution notes:

hostedcontrolplane_controller.go: The v1 reconcileOLMAndMiscCerts function was refactored out in release-4.22. The scheduler cert reconciliation was added directly to the existing reconcilePKI function instead.

kube_scheduler/component.go: Kept util.AvailabilityProberOpts{} (release-4.22) instead of podspec.AvailabilityProberOpts{} (main), while adding the ServiceMonitor manifest adapter.

Which issue(s) this PR fixes:

Fixes CNTRLPLANE-4314

Special notes for your reviewer:

The conflict resolution in hostedcontrolplane_controller.go is the key area to review — the scheduler cert was placed between KCM and CVO certs in reconcilePKI, matching the logical placement from the original PR.

PR CNTRLPLANE-3954: add kube-scheduler metrics endpoint e2e coverage #9159 (e2e tests) was intentionally not backported because it gates on Version423 (would be skipped on 4.22) and depends on test/e2e/v2/util which doesn't exist on this branch.

Checklist:

Subject and description added to both, commit and PR.

Relevant issues have been referenced.

This change includes docs.

This change includes unit tests.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

celebdor · 2026-09-03T07:54:35Z

/jira refresh

openshift-ci-robot · 2026-09-03T07:54:38Z

@celebdor: This pull request references CNTRLPLANE-4321 which is a valid jira issue.

Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

muraee · 2026-09-03T09:36:34Z

/approve

openshift-ci · 2026-09-03T09:37:37Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: michaelryanmcneill, muraee

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [muraee]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

jparrill · 2026-09-03T13:53:16Z

/retest-required

jparrill · 2026-09-03T13:53:59Z

/lgtm

openshift-merge-bot · 2026-09-03T13:54:21Z

Scheduling tests matching the pipeline_run_if_changed or not excluded by pipeline_skip_if_only_changed parameters:
/test e2e-aks
/test e2e-azure-aks-external-oidc
/test e2e-azure-self-managed
/test e2e-kubevirt-aws-ovn-reduced

michaelryanmcneill · 2026-09-03T14:00:10Z

/jira refresh

openshift-ci-robot · 2026-09-03T14:00:16Z

@michaelryanmcneill: This pull request references CNTRLPLANE-4321 which is a valid jira issue.

Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

michaelryanmcneill · 2026-09-03T18:05:03Z

/test e2e-aws-upgrade-hypershift-operator

michaelryanmcneill · 2026-09-03T22:02:57Z

/test e2e-aws-upgrade-hypershift-operator

michaelryanmcneill · 2026-09-04T14:44:55Z

/retest-required

csrwng · 2026-09-04T19:14:00Z

/override ci/prow/e2e-aws-upgrade-hypershift-operator

openshift-ci · 2026-09-04T19:14:26Z

@csrwng: Overrode contexts on behalf of csrwng: ci/prow/e2e-aws-upgrade-hypershift-operator

Details

In response to this:

/override ci/prow/e2e-aws-upgrade-hypershift-operator

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

michaelryanmcneill · 2026-09-04T19:16:00Z

/jira refresh

openshift-ci-robot · 2026-09-04T19:16:04Z

@michaelryanmcneill: This pull request references CNTRLPLANE-4321 which is a valid jira issue.

Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci · 2026-09-04T19:19:19Z

@michaelryanmcneill: The label(s) jira/valid-bug cannot be applied or removed, because you are not in one of the allowed teams and are not an allowed user. Must be a member of one of these teams: openshift-patch-managers, openshift-staff-engineers, openshift-release-oversight, openshift-sustaining-engineers

Details

In response to this:

/label jira/valid-bug

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

michaelryanmcneill · 2026-09-04T19:20:07Z

/retitle [release-4.22] CNTRLPLANE-4321: Cherry-pick kube-scheduler metrics support

michaelryanmcneill · 2026-09-04T19:20:57Z

/jira refresh

openshift-ci-robot · 2026-09-04T19:21:00Z

@michaelryanmcneill: This pull request references CNTRLPLANE-4321 which is a valid jira issue.

Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

michaelryanmcneill · 2026-09-04T19:27:07Z

/refresh

openshift-ci · 2026-09-04T19:41:28Z

@michaelryanmcneill: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-aws-upgrade-hypershift-operator	`57bcd6d`	link	true	`/test e2e-aws-upgrade-hypershift-operator`

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

dhruv-gautam added 3 commits September 2, 2026 13:02

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Sep 2, 2026

openshift-ci Bot added the do-not-merge/needs-area label Sep 2, 2026

openshift-ci Bot requested review from jparrill and muraee September 2, 2026 17:20

openshift-ci Bot added the area/control-plane-operator Indicates the PR includes changes for the control plane operator - in an OCP release label Sep 2, 2026

openshift-ci Bot added area/hypershift-operator Indicates the PR includes changes for the hypershift operator and API - outside an OCP release and removed do-not-merge/needs-area labels Sep 2, 2026

michaelryanmcneill closed this Sep 2, 2026

michaelryanmcneill reopened this Sep 2, 2026

openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label Sep 2, 2026

celebdor changed the title ~~CNTRLPLANE-4314: Cherry-pick kube-scheduler metrics support to release-4.22~~ CNTRLPLANE-4321: Cherry-pick kube-scheduler metrics support to release-4.22 Sep 3, 2026

openshift-ci Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 3, 2026

openshift-ci Bot assigned jparrill Sep 3, 2026

openshift-ci Bot added the lgtm Indicates that a PR is ready to be merged. label Sep 3, 2026

csrwng added the backport-risk-assessed Indicates a PR to a release branch has been evaluated and considered safe to accept. label Sep 4, 2026

openshift-ci Bot changed the title ~~CNTRLPLANE-4321: Cherry-pick kube-scheduler metrics support to release-4.22~~ [release-4.22] CNTRLPLANE-4321: Cherry-pick kube-scheduler metrics support Sep 4, 2026

csrwng added the jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. label Sep 4, 2026

openshift-merge-bot Bot merged commit 7273000 into openshift:release-4.22 Sep 4, 2026
31 of 32 checks passed

Uh oh!

Conversation

michaelryanmcneill commented Sep 2, 2026

What this PR does / why we need it:

Conflict resolution notes:

Which issue(s) this PR fixes:

Special notes for your reviewer:

Checklist:

Uh oh!

openshift-merge-bot Bot commented Sep 2, 2026

Uh oh!

openshift-ci-robot commented Sep 2, 2026 • edited by openshift-ci Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does / why we need it:

Conflict resolution notes:

Which issue(s) this PR fixes:

Special notes for your reviewer:

Checklist:

Uh oh!

coderabbitai Bot commented Sep 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

michaelryanmcneill commented Sep 2, 2026

Uh oh!

openshift-ci Bot commented Sep 2, 2026

Uh oh!

michaelryanmcneill commented Sep 2, 2026

Uh oh!

codecov Bot commented Sep 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

michaelryanmcneill commented Sep 2, 2026

Uh oh!

michaelryanmcneill commented Sep 2, 2026

Uh oh!

michaelryanmcneill commented Sep 2, 2026

Uh oh!

openshift-ci-robot commented Sep 2, 2026

Uh oh!

openshift-ci-robot commented Sep 3, 2026 • edited by openshift-ci Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What this PR does / why we need it:

Conflict resolution notes:

Which issue(s) this PR fixes:

Special notes for your reviewer:

Checklist:

Uh oh!

celebdor commented Sep 3, 2026

Uh oh!

openshift-ci-robot commented Sep 3, 2026 • edited by openshift-ci Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

muraee commented Sep 3, 2026

Uh oh!

openshift-ci Bot commented Sep 3, 2026

Uh oh!

jparrill commented Sep 3, 2026

Uh oh!

jparrill commented Sep 3, 2026

Uh oh!

openshift-merge-bot Bot commented Sep 3, 2026

Uh oh!

michaelryanmcneill commented Sep 3, 2026

Uh oh!

openshift-ci-robot commented Sep 3, 2026 • edited by openshift-ci Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

michaelryanmcneill commented Sep 3, 2026

Uh oh!

michaelryanmcneill commented Sep 3, 2026

Uh oh!

michaelryanmcneill commented Sep 4, 2026

Uh oh!

csrwng commented Sep 4, 2026

Uh oh!

openshift-ci Bot commented Sep 4, 2026

Uh oh!

michaelryanmcneill commented Sep 4, 2026

openshift-ci-robot commented Sep 2, 2026 •

edited by openshift-ci Bot

Loading

coderabbitai Bot commented Sep 2, 2026 •

edited

Loading

codecov Bot commented Sep 2, 2026 •

edited

Loading

openshift-ci-robot commented Sep 3, 2026 •

edited by openshift-ci Bot

Loading

openshift-ci-robot commented Sep 3, 2026 •

edited by openshift-ci Bot

Loading

openshift-ci-robot commented Sep 3, 2026 •

edited by openshift-ci Bot

Loading

openshift-ci-robot commented Sep 4, 2026 •

edited by openshift-ci Bot

Loading

michaelryanmcneill commented Sep 4, 2026 •

edited by openshift-ci Bot

Loading

openshift-ci-robot commented Sep 4, 2026 •

edited by openshift-ci Bot

Loading