OCPBUGS-54877: Add catalogd-cas-dir option to op-con #316

tmshort · 2025-04-15T16:10:18Z

This forces us to explicitly watch the /var/ca-certs dir, and properly reload the certs when the x509.SystemCertPool() may not.

Signed-off-by: Todd Short <todd.short@me.com>

openshift-ci · 2025-04-15T16:17:17Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: tmshort

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~DOWNSTREAM_OWNERS~~ [tmshort]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

grokspawn · 2025-04-15T16:42:51Z

...rator-controller/kustomize/overlays/openshift/olmv1-ns/patches/manager_deployment_certs.yaml

 - op: add
  path: /spec/template/spec/containers/0/env
  value: [{"name":"SSL_CERT_DIR", "value":"/var/ca-certs"}]


Kinda feel like we should take this out of the manifests since we're providing a nearly-equivalent (but better) option as part of this PR.
If SSL_CERT_FILE is being used for container/images, then does it need to also be updated to be able to read updates to those paths successfully?
Because of https://github.com/golang/go/blob/master/src/crypto/x509/cert_pool.go?name=release#L116 ?

I'm reluctant to, since it's still being used as a base for the SystemCertPool, and thus used by container/images.

And setting the --pull-cas-dir completely changes the behavior (we lose additionalTrustedCA support)
And I just realized, the SSL_CERTS_DIR is needed for running our upstream e2e testing using a local repo that is a local service

I feel like we should copy some of this out-of-codebase, out-of-mind knowledge into this file as comments to explain to our future selves:

What the heck all this is for

Any gotchas that we know about, so that we avoid tripping over them in the future

If folks agree with that, it'd be maybe better in a follow-up once we've got it all figured out and we're not in fire drill mode.

There's already something documented in Google Docs.

My main thing is having a breadcrumb here that helps us make sense if what's here. Can we link the google doc here?

Yes... but followup?

openshift-ci-robot · 2025-04-15T16:50:30Z

@tmshort: This pull request explicitly references no jira issue.

In response to this:

This forces us to explicitly watch the /var/ca-certs dir, and properly reload the certs when the x509.SystemCertPool() may not.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

tmshort · 2025-04-15T16:53:06Z

/payload-aggregate periodic-ci-openshift-multiarch-master-nightly-4.19-ocp-e2e-azure-ovn-multi-x-ax 10

openshift-ci · 2025-04-15T16:53:09Z

@tmshort: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

periodic-ci-openshift-multiarch-master-nightly-4.19-ocp-e2e-azure-ovn-multi-x-ax

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/1b33d920-1a1a-11f0-8c5f-ee739d333a54-0

tmshort · 2025-04-15T16:53:14Z

/payload-aggregate periodic-ci-openshift-release-master-nightly-4.19-e2e-vsphere-ovn 10

openshift-ci · 2025-04-15T16:53:18Z

@tmshort: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

periodic-ci-openshift-release-master-nightly-4.19-e2e-vsphere-ovn

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/1fb394e0-1a1a-11f0-8220-72a13906a7d1-0

anik120 · 2025-04-15T17:42:55Z

/retitle OCPBUGS-54877: Add catalogd-cas-dir option to op-con

openshift-ci-robot · 2025-04-15T17:43:36Z

@tmshort: This pull request references Jira Issue OCPBUGS-54877, which is valid.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target version (4.19.0) matches configured target version for branch (4.19.0)
bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @Xia-Zhao-rh

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

This forces us to explicitly watch the /var/ca-certs dir, and properly reload the certs when the x509.SystemCertPool() may not.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

anik120 · 2025-04-15T17:47:16Z

the e2e-aws-techpreview job failed but I don't see the "x509" error, which was reliably there in the previous failures. Retesting after a while since both e2e-aws and e2e-techpreview failed with:

install should succeed: infrastructure 
-- | --
{  openshift cluster install failed with infrastructure setup}

which looks like aws cluster provisioning issue

tmshort · 2025-04-15T18:46:24Z

/retest-required
It's been an hour

tmshort · 2025-04-15T18:47:05Z

/test openshift-e2e-aws-techpreview

oceanc80 · 2025-04-15T20:39:09Z

/test openshift-e2e-aws

tmshort · 2025-04-15T21:21:46Z

/payload-aggregate-with-prs periodic-ci-openshift-multiarch-master-nightly-4.19-ocp-e2e-azure-ovn-multi-x-ax 5 #316

openshift-ci · 2025-04-15T21:21:49Z

@tmshort: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

periodic-ci-openshift-multiarch-master-nightly-4.19-ocp-e2e-azure-ovn-multi-x-ax

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/a34b5390-1a3f-11f0-9ec5-1d03e7a8039c-0

openshift-ci · 2025-04-15T21:21:50Z

@tmshort: This PR was included in a payload test run from #316
trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

periodic-ci-openshift-multiarch-master-nightly-4.19-ocp-e2e-azure-ovn-multi-x-ax

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/a34b5390-1a3f-11f0-9ec5-1d03e7a8039c-0

tmshort · 2025-04-15T21:22:06Z

/payload-aggregate periodic-ci-openshift-release-master-nightly-4.19-e2e-vsphere-ovn 10

openshift-ci · 2025-04-15T21:22:09Z

@tmshort: trigger 1 job(s) for the /payload-(with-prs|job|aggregate|job-with-prs|aggregate-with-prs) command

periodic-ci-openshift-release-master-nightly-4.19-e2e-vsphere-ovn

See details on https://pr-payload-tests.ci.openshift.org/runs/ci/af65c6b0-1a3f-11f0-9286-4615665673f7-0

joelanford · 2025-04-16T00:25:58Z

/retest-required

jianzhangbjz · 2025-04-16T00:41:17Z

Trigger a test via the cluster-bot:

workflow-launch cucushift-agent-vsphere-install-ha #316
a cluster is being created - I'll send you the credentials when the cluster is ready.

oceanc80 · 2025-04-16T01:29:17Z

/test openshift-e2e-aws-techpreview

tmshort · 2025-04-16T03:19:20Z

The periodic-ci-openshift-multiarch-master-nightly-4.19-ocp-e2e-azure-ovn-multi-x-ax tests do not seem to be running with this PR316. The --catalogd-cas-dir setting is not on the operator-controller deployment.
The periodic-ci-openshift-release-master-nightly-4.19-e2e-vsphere-ovn tests do seem to be running with this PR316, however, they seem to be based on 4.19.0-0.nightly-2025-04-10-031621 and not on something more recent...

jianzhangbjz · 2025-04-16T03:28:34Z

Test pass, see: https://issues.redhat.com/browse/OCPBUGS-54877
/label qe-approved
/lgtm

openshift-ci-robot · 2025-04-16T03:28:41Z

@tmshort: This pull request references Jira Issue OCPBUGS-54877, which is valid.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target version (4.19.0) matches configured target version for branch (4.19.0)
bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

No GitHub users were found matching the public email listed for the QA contact in Jira (jiazha@redhat.com), skipping review request.

In response to this:

This forces us to explicitly watch the /var/ca-certs dir, and properly reload the certs when the x509.SystemCertPool() may not.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2025-04-16T05:54:46Z

/retest-required

Remaining retests: 0 against base HEAD 209ff44 and 2 for PR HEAD ae69917 in total

openshift-ci · 2025-04-16T08:56:32Z

@tmshort: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/okd-scos-e2e-aws-ovn	`ae69917`	link	false	`/test okd-scos-e2e-aws-ovn`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

openshift-ci-robot · 2025-04-16T09:04:30Z

@tmshort: Jira Issue OCPBUGS-54877: Some pull requests linked via external trackers have merged:

openshift/cluster-olm-operator#114

The following pull requests linked via external trackers have not merged:

openshift/operator-framework-operator-controller#311 is open

These pull request must merge or be unlinked from the Jira bug in order for it to move to the next state. Once unlinked, request a bug refresh with /jira refresh.

Jira Issue OCPBUGS-54877 has not been moved to the MODIFIED state.

In response to this:

This forces us to explicitly watch the /var/ca-certs dir, and properly reload the certs when the x509.SystemCertPool() may not.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-bot · 2025-04-16T13:53:34Z

[ART PR BUILD NOTIFIER]

Distgit: ose-olm-catalogd
This PR has been included in build ose-olm-catalogd-container-v4.19.0-202504161306.p0.gad55882.assembly.stream.el9.
All builds following this will include this PR.

openshift-bot · 2025-04-16T13:56:34Z

[ART PR BUILD NOTIFIER]

Distgit: ose-olm-operator-controller
This PR has been included in build ose-olm-operator-controller-container-v4.19.0-202504161306.p0.gad55882.assembly.stream.el9.
All builds following this will include this PR.

tmshort · 2025-04-21T13:15:06Z

/cherry-pick release-4.18

openshift-cherrypick-robot · 2025-04-21T13:15:48Z

@tmshort: #316 failed to apply on top of branch "release-4.18":

Applying: UPSTREAM: <carry>: Add caalogd-cas-dir option to op-con
Using index info to reconstruct a base tree...
A	openshift/operator-controller/kustomize/overlays/openshift/olmv1-ns/patches/manager_deployment_certs.yaml
A	openshift/operator-controller/manifests/20-deployment-openshift-operator-controller-operator-controller-controller-manager.yml
Falling back to patching base and 3-way merge...
Auto-merging openshift/manifests/20-deployment-openshift-operator-controller-operator-controller-controller-manager.yml
CONFLICT (content): Merge conflict in openshift/manifests/20-deployment-openshift-operator-controller-operator-controller-controller-manager.yml
Auto-merging openshift/kustomize/overlays/openshift/olmv1-ns/patches/manager_deployment_certs.yaml
CONFLICT (content): Merge conflict in openshift/kustomize/overlays/openshift/olmv1-ns/patches/manager_deployment_certs.yaml
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config advice.mergeConflict false"
Patch failed at 0001 UPSTREAM: <carry>: Add caalogd-cas-dir option to op-con

In response to this:

/cherry-pick release-4.18

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

tmshort · 2025-04-21T13:16:19Z

Will have to cherry-pick manually.

UPSTREAM: <carry>: Add caalogd-cas-dir option to op-con

ae69917

Signed-off-by: Todd Short <todd.short@me.com>

openshift-ci bot requested review from camilamacedo86 and grokspawn April 15, 2025 16:16

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 15, 2025

tmshort changed the title ~~UPSTREAM: <carry>: Add caalogd-cas-dir option to op-con~~ UPSTREAM: <carry>: Add catalogd-cas-dir option to op-con Apr 15, 2025

grokspawn reviewed Apr 15, 2025

View reviewed changes

tmshort changed the title ~~UPSTREAM: <carry>: Add catalogd-cas-dir option to op-con~~ NO-ISSUE: Add catalogd-cas-dir option to op-con Apr 15, 2025

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Apr 15, 2025

openshift-ci bot changed the title ~~NO-ISSUE: Add catalogd-cas-dir option to op-con~~ OCPBUGS-54877: Add catalogd-cas-dir option to op-con Apr 15, 2025

openshift-ci-robot added the jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. label Apr 15, 2025

openshift-ci bot requested a review from Xia-Zhao-rh April 15, 2025 17:43

openshift-ci bot added the qe-approved Signifies that QE has signed off on this PR label Apr 16, 2025

openshift-ci bot assigned jianzhangbjz Apr 16, 2025

openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Apr 16, 2025

openshift-merge-bot bot merged commit ad55882 into openshift:main Apr 16, 2025
10 of 11 checks passed

tmshort deleted the update-cert-config branch April 16, 2025 14:55

OCPBUGS-54877: Add catalogd-cas-dir option to op-con #316

OCPBUGS-54877: Add catalogd-cas-dir option to op-con #316

Uh oh!

Conversation

tmshort commented Apr 15, 2025

Uh oh!

openshift-ci bot commented Apr 15, 2025

Uh oh!

grokspawn Apr 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tmshort Apr 15, 2025

Choose a reason for hiding this comment

Uh oh!

tmshort Apr 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joelanford Apr 15, 2025

Choose a reason for hiding this comment

Uh oh!

tmshort Apr 15, 2025

Choose a reason for hiding this comment

Uh oh!

joelanford Apr 15, 2025

Choose a reason for hiding this comment

Uh oh!

tmshort Apr 16, 2025

Choose a reason for hiding this comment

Uh oh!

openshift-ci-robot commented Apr 15, 2025

Uh oh!

tmshort commented Apr 15, 2025

Uh oh!

openshift-ci bot commented Apr 15, 2025

Uh oh!

tmshort commented Apr 15, 2025

Uh oh!

openshift-ci bot commented Apr 15, 2025

Uh oh!

anik120 commented Apr 15, 2025

Uh oh!

openshift-ci-robot commented Apr 15, 2025

Uh oh!

anik120 commented Apr 15, 2025

Uh oh!

tmshort commented Apr 15, 2025

Uh oh!

tmshort commented Apr 15, 2025

Uh oh!

oceanc80 commented Apr 15, 2025

Uh oh!

tmshort commented Apr 15, 2025

Uh oh!

openshift-ci bot commented Apr 15, 2025

Uh oh!

openshift-ci bot commented Apr 15, 2025

Uh oh!

tmshort commented Apr 15, 2025

Uh oh!

openshift-ci bot commented Apr 15, 2025

Uh oh!

joelanford commented Apr 16, 2025

Uh oh!

jianzhangbjz commented Apr 16, 2025

Uh oh!

oceanc80 commented Apr 16, 2025

Uh oh!

tmshort commented Apr 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jianzhangbjz commented Apr 16, 2025

Uh oh!

openshift-ci-robot commented Apr 16, 2025

Uh oh!

openshift-ci-robot commented Apr 16, 2025

Uh oh!

openshift-ci bot commented Apr 16, 2025

Uh oh!

grokspawn Apr 15, 2025 •

edited

Loading

tmshort Apr 15, 2025 •

edited

Loading

tmshort commented Apr 16, 2025 •

edited

Loading