OCPBUGS-86251: fix: TLS client cert rotation when no CA is configured

[machine424]

Make sure it doesn't panic AND doesn't perma-fail.

Point prometheus/common to a patched version that
contains prometheus/common#908

Add TestClientTLSCertRotationWithoutCAFile (backported from prometheus#18727) to verify the fix end-to-end.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: e86b3b3b-59a7-4be2-a749-a94579a8b8f1

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[machine424]

--- BEFORE FIX (unpatched prometheus/common v0.65.0) ---

=== RUN   TestClientTLSCertRotationWithoutCAFile
=== PAUSE TestClientTLSCertRotationWithoutCAFile
=== CONT  TestClientTLSCertRotationWithoutCAFile
    reload_test.go:194: time=2026-05-20T11:08:41.267+02:00 level=INFO source=main.go:1551 msg="updated GOGC" old=100 new=75
    reload_test.go:194: time=2026-05-20T11:08:41.267+02:00 level=INFO source=main.go:680 msg="Leaving GOMAXPROCS=12: CPU quota undefined" component=automaxprocs
    reload_test.go:194: time=2026-05-20T11:08:41.267+02:00 level=INFO source=memlimit.go:198 msg="GOMEMLIMIT is updated" component=automemlimit package=github.com/KimMachineGun/automemlimit/memlimit GOMEMLIMIT=30923764531 previous=9223372036854775807
    reload_test.go:194: time=2026-05-20T11:08:41.267+02:00 level=INFO source=main.go:722 msg="No time or size retention was set so using the default time retention" duration=15d
    reload_test.go:194: time=2026-05-20T11:08:41.267+02:00 level=INFO source=main.go:776 msg="Starting Prometheus Server" mode=server version="(version=, branch=, revision=unknown)"
    reload_test.go:194: time=2026-05-20T11:08:41.267+02:00 level=INFO source=main.go:781 msg="operational information" build_context="(go=go1.25.0, platform=darwin/arm64, user=, date=, tags=builtinassets)" host_details=(darwin) fd_limits="(soft=122880, hard=unlimited)" vm_limits="(soft=unlimited, hard=unlimited)"
    reload_test.go:194: time=2026-05-20T11:08:41.270+02:00 level=INFO source=web.go:656 msg="Start listening for connections" component=web address=0.0.0.0:59582
    reload_test.go:194: time=2026-05-20T11:08:41.270+02:00 level=INFO source=main.go:1295 msg="Starting TSDB ..."
    reload_test.go:194: time=2026-05-20T11:08:41.271+02:00 level=INFO source=tls_config.go:347 msg="Listening on" component=web address=[::]:59582
    reload_test.go:194: time=2026-05-20T11:08:41.271+02:00 level=INFO source=tls_config.go:350 msg="TLS is disabled." component=web http2=false address=[::]:59582
    reload_test.go:194: time=2026-05-20T11:08:41.273+02:00 level=INFO source=head.go:657 msg="Replaying on-disk memory mappable chunks if any" component=tsdb
    reload_test.go:194: time=2026-05-20T11:08:41.273+02:00 level=INFO source=head.go:744 msg="On-disk memory mappable chunks replay completed" component=tsdb duration=500ns
    reload_test.go:194: time=2026-05-20T11:08:41.273+02:00 level=INFO source=head.go:752 msg="Replaying WAL, this may take a while" component=tsdb
    reload_test.go:194: time=2026-05-20T11:08:41.274+02:00 level=INFO source=head.go:825 msg="WAL segment loaded" component=tsdb segment=0 maxSegment=0 duration=239.041µs
    reload_test.go:194: time=2026-05-20T11:08:41.274+02:00 level=INFO source=head.go:862 msg="WAL replay completed" component=tsdb checkpoint_replay_duration=75.166µs wal_replay_duration=257µs wbl_replay_duration=84ns chunk_snapshot_load_duration=0s mmap_chunk_replay_duration=500ns total_replay_duration=341.375µs
    reload_test.go:194: time=2026-05-20T11:08:41.275+02:00 level=INFO source=main.go:1316 msg="filesystem information" fs_type=1a
    reload_test.go:194: time=2026-05-20T11:08:41.275+02:00 level=INFO source=main.go:1319 msg="TSDB started"
    reload_test.go:194: time=2026-05-20T11:08:41.275+02:00 level=INFO source=main.go:1504 msg="Loading configuration file" filename=/var/folders/pl/4zxvwhbd0q5bkw9vxnk08swm0000gn/T/TestClientTLSCertRotationWithoutCAFile1634498521/001/prometheus.yml
    reload_test.go:194: time=2026-05-20T11:08:41.278+02:00 level=INFO source=main.go:1544 msg="Completed loading of configuration file" db_storage=792ns remote_storage=15.583µs web_handler=292ns query_engine=500ns scrape=2.689292ms scrape_sd=16.875µs notify=1.042µs notify_sd=458ns rules=1.25µs tracing=3.417µs filename=/var/folders/pl/4zxvwhbd0q5bkw9vxnk08swm0000gn/T/TestClientTLSCertRotationWithoutCAFile1634498521/001/prometheus.yml totalDuration=3.16ms
    reload_test.go:194: time=2026-05-20T11:08:41.278+02:00 level=INFO source=main.go:1280 msg="Server is ready to receive web requests."
    reload_test.go:194: time=2026-05-20T11:08:41.278+02:00 level=INFO source=manager.go:176 msg="Starting rule manager..." component="rule manager"
    reload_test.go:194: panic: runtime error: invalid memory address or nil pointer dereference
    reload_test.go:194: [signal SIGSEGV: segmentation violation code=0x2 addr=0x18 pc=0x1007920fc]
    reload_test.go:194: 
    reload_test.go:194: goroutine 618 [running]:
    reload_test.go:194: github.com/prometheus/common/config.(*tlsRoundTripper).RoundTrip(0x1400102a000, 0x140016048c0)
    reload_test.go:194: 	$PROMETHEUS_ROOT/vendor/github.com/prometheus/common/config/http_config.go:1380 +0x40c
    reload_test.go:194: net/http.send(0x140016048c0, {0x1045566c0, 0x1400102a000}, {0x10055c188?, 0x8?, 0x0?})
    reload_test.go:194: 	/opt/homebrew/Cellar/go/1.25.0/libexec/src/net/http/client.go:259 +0x478
    reload_test.go:194: net/http.(*Client).send(0x14000fa3590, 0x140016048c0, {0x140010e1818?, 0x100292ca8?, 0x0?})
    reload_test.go:194: 	/opt/homebrew/Cellar/go/1.25.0/libexec/src/net/http/client.go:180 +0x88
    reload_test.go:194: net/http.(*Client).do(0x14000fa3590, 0x140016048c0)
    reload_test.go:194: 	/opt/homebrew/Cellar/go/1.25.0/libexec/src/net/http/client.go:729 +0x710
    reload_test.go:194: net/http.(*Client).Do(...)
    reload_test.go:194: 	/opt/homebrew/Cellar/go/1.25.0/libexec/src/net/http/client.go:587
    reload_test.go:194: github.com/prometheus/prometheus/scrape.(*targetScraper).scrape(0x140013c4840, {0x10459fd20, 0x140016018f0})
    reload_test.go:194: 	$PROMETHEUS_ROOT/scrape/scrape.go:836 +0x4e8
    reload_test.go:194: github.com/prometheus/prometheus/scrape.(*scrapeLoop).scrapeAndReport(0x14000f7e160, {0xee19f7120?, 0x106929b40?, 0x106929b40?}, {0xee19f7120?, 0x106929b40?, 0x106929b40?}, 0x0)
    reload_test.go:194: 	$PROMETHEUS_ROOT/scrape/scrape.go:1448 +0x294
    reload_test.go:194: github.com/prometheus/prometheus/scrape.(*scrapeLoop).run(0x14000f7e160, 0x0)
    reload_test.go:194: 	$PROMETHEUS_ROOT/scrape/scrape.go:1373 +0x250
    reload_test.go:194: created by github.com/prometheus/prometheus/scrape.(*scrapePool).sync in goroutine 617
    reload_test.go:194: 	$PROMETHEUS_ROOT/scrape/scrape.go:612 +0x8e8
    main_test.go:1098: 
        	Error Trace:	$PROMETHEUS_ROOT/cmd/prometheus/main_test.go:1098
        	Error:      	Condition never satisfied
        	Test:       	TestClientTLSCertRotationWithoutCAFile
        	Messages:   	scrape should keep working after cert rotation
--- FAIL: TestClientTLSCertRotationWithoutCAFile (11.52s)
FAIL
FAIL	github.com/prometheus/prometheus/cmd/prometheus	12.966s
FAIL

[machine424]

/jira cherrypick OCPBUGS-86250

[openshift-ci-robot]

@machine424: Jira Issue OCPBUGS-86250 has been cloned as Jira Issue OCPBUGS-86251. Will retitle bug to link to clone.
/retitle OCPBUGS-86251: fix: TLS client cert rotation when no CA is configured

Details

In response to this:

/jira cherrypick OCPBUGS-86250

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@machine424: This pull request references Jira Issue OCPBUGS-86251, which is invalid:

• expected dependent Jira Issue OCPBUGS-86250 to be in one of the following states: VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA), but it is New instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Make sure it doesn't panic AND doesn't perma-fail.

Point prometheus/common to a patched version that
contains prometheus/common#908

Add TestClientTLSCertRotationWithoutCAFile (backported from prometheus#18727) to verify the fix end-to-end.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[machine424]

/jira refresh

[openshift-ci-robot]

@machine424: This pull request references Jira Issue OCPBUGS-86251, which is invalid:

• expected dependent Jira Issue OCPBUGS-86250 to be in one of the following states: VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA), but it is New instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

Details

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[machine424]

/label backport-risk-assessed

[machine424]

/verified by added test

[openshift-ci-robot]

@machine424: This PR has been marked as verified by added test.

Details

In response to this:

/verified by added test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[danielmellado]

/lgtm

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: danielmellado, machine424

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [danielmellado,machine424]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

@machine424: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.