Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

oidcdiscoveryendpoint: add status handler #220

Merged
merged 1 commit into from Jul 9, 2020
Merged

oidcdiscoveryendpoint: add status handler #220

merged 1 commit into from Jul 9, 2020

Conversation

sjenning
Copy link
Contributor

@sjenning sjenning commented Jul 2, 2020

/cc @joelddiaz
/assign @dgoodwin

Following #213, this PR adds a status handler to the oidcdiscoveryendpoint controller so that if it is having problems, the overall OperatorStatus is Degraded with informative message why.

fyi @derekwaynecarr

@sjenning sjenning force-pushed the add-oidc-controller-status-handler branch 2 times, most recently from 4385550 to fd0fd11 Compare July 2, 2020 17:19
@sjenning
Copy link
Contributor Author

sjenning commented Jul 2, 2020

Looks good in my test cluster. AWS ids appear in some error messages though... might need to not include those to avoid unneeded updates to the OperatorStatus.

$ oc logs cloud-credential-operator-5dc878f9d-jwjjc -c cloud-credential-operator  | grep oidc
time="2020-07-02T17:38:10Z" level=info msg="reconciling AWS S3 OIDC discovery endpoint" controller=oidcdiscoveryendpoint
time="2020-07-02T17:38:10Z" level=error msg="failed reconciling S3 resources" controller=oidcdiscoveryendpoint error="secrets \"cloud-credential-operator-s3-creds\" not found"
time="2020-07-02T17:38:10Z" level=info msg="syncing cluster operator status" controller=oidcdiscoveryendpoint
time="2020-07-02T17:38:10Z" level=info msg="cluster operator status updated" controller=oidcdiscoveryendpoint <-- Degraded is True
time="2020-07-02T17:38:12Z" level=info msg="reconciling AWS S3 OIDC discovery endpoint" controller=oidcdiscoveryendpoint
time="2020-07-02T17:38:12Z" level=error msg="failed reconciling S3 resources" controller=oidcdiscoveryendpoint error="secrets \"cloud-credential-operator-s3-creds\" not found"
time="2020-07-02T17:38:13Z" level=info msg="reconciling AWS S3 OIDC discovery endpoint" controller=oidcdiscoveryendpoint
time="2020-07-02T17:38:13Z" level=error msg="failed reconciling S3 resources" controller=oidcdiscoveryendpoint error="secrets \"cloud-credential-operator-s3-creds\" not found"
time="2020-07-02T17:38:14Z" level=info msg="reconciling AWS S3 OIDC discovery endpoint" controller=oidcdiscoveryendpoint
time="2020-07-02T17:38:14Z" level=error msg="failed reconciling S3 resources" controller=oidcdiscoveryendpoint error="secrets \"cloud-credential-operator-s3-creds\" not found"
time="2020-07-02T17:38:15Z" level=info msg="reconciling AWS S3 OIDC discovery endpoint" controller=oidcdiscoveryendpoint
time="2020-07-02T17:38:15Z" level=error msg="failed reconciling S3 resources" controller=oidcdiscoveryendpoint error="secrets \"cloud-credential-operator-s3-creds\" not found"
time="2020-07-02T17:38:16Z" level=info msg="reconciling AWS S3 OIDC discovery endpoint" controller=oidcdiscoveryendpoint
time="2020-07-02T17:38:16Z" level=error msg="failed reconciling S3 resources" controller=oidcdiscoveryendpoint error="secrets \"cloud-credential-operator-s3-creds\" not found"
time="2020-07-02T17:38:17Z" level=info msg="reconciling AWS S3 OIDC discovery endpoint" controller=oidcdiscoveryendpoint
time="2020-07-02T17:38:17Z" level=error msg="failed reconciling S3 resources" controller=oidcdiscoveryendpoint error="secrets \"cloud-credential-operator-s3-creds\" not found"
time="2020-07-02T17:38:18Z" level=info msg="reconciling AWS S3 OIDC discovery endpoint" controller=oidcdiscoveryendpoint
time="2020-07-02T17:38:18Z" level=error msg="failed reconciling S3 resources" controller=oidcdiscoveryendpoint error="secrets \"cloud-credential-operator-s3-creds\" not found"
time="2020-07-02T17:38:19Z" level=info msg="reconciling AWS S3 OIDC discovery endpoint" controller=oidcdiscoveryendpoint
time="2020-07-02T17:38:19Z" level=error msg="failed reconciling S3 resources" controller=oidcdiscoveryendpoint error="secrets \"cloud-credential-operator-s3-creds\" not found"
time="2020-07-02T17:38:20Z" level=info msg="reconciling AWS S3 OIDC discovery endpoint" controller=oidcdiscoveryendpoint
time="2020-07-02T17:38:20Z" level=error msg="failed reconciling S3 resources" controller=oidcdiscoveryendpoint error="secrets \"cloud-credential-operator-s3-creds\" not found"
time="2020-07-02T17:38:22Z" level=info msg="reconciling AWS S3 OIDC discovery endpoint" controller=oidcdiscoveryendpoint
time="2020-07-02T17:38:23Z" level=error msg="failed reconciling S3 resources" controller=oidcdiscoveryendpoint error="InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.\n\tstatus code: 403, request id: 2F1D8F27348C2344, host id: lCHMJlRi+VRGVSl+ozeGrU14N8CHmSk+ne4m6C6gpAg8bGUHmC2eVp4qJVagpHdArZxAIFykkHo="
time="2020-07-02T17:38:23Z" level=info msg="syncing cluster operator status" controller=oidcdiscoveryendpoint
time="2020-07-02T17:38:23Z" level=info msg="cluster operator status updated" controller=oidcdiscoveryendpoint <-- Degraded is still True but for different reason
time="2020-07-02T17:38:28Z" level=info msg="reconciling AWS S3 OIDC discovery endpoint" controller=oidcdiscoveryendpoint
time="2020-07-02T17:38:28Z" level=error msg="failed reconciling S3 resources" controller=oidcdiscoveryendpoint error="InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.\n\tstatus code: 403, request id: 1C829F2102850A86, host id: +S3rJ6Pxbe5+MTWfNQ6VegWA/jBJDejVxQ2ujLbTbvQrC5dLPKAPQ3bgvgTAhFMAWM7aKU2rp00="
time="2020-07-02T17:38:28Z" level=info msg="syncing cluster operator status" controller=oidcdiscoveryendpoint
time="2020-07-02T17:38:28Z" level=info msg="cluster operator status updated" controller=oidcdiscoveryendpoint <-- Unfortunate update due to changing ids in error message :(
time="2020-07-02T17:38:38Z" level=info msg="reconciling AWS S3 OIDC discovery endpoint" controller=oidcdiscoveryendpoint
time="2020-07-02T17:38:39Z" level=info msg="bucket created" bucket=sjenning-aws-8f6j4-oidc controller=oidcdiscoveryendpoint
time="2020-07-02T17:38:39Z" level=info msg="syncing cluster operator status" controller=oidcdiscoveryendpoint
time="2020-07-02T17:38:39Z" level=info msg="cluster operator status updated" controller=oidcdiscoveryendpoint <-- Degraded switched to false

@sjenning sjenning force-pushed the add-oidc-controller-status-handler branch 2 times, most recently from 374feac to 541f4eb Compare July 2, 2020 20:06
@sjenning
Copy link
Contributor Author

sjenning commented Jul 6, 2020

/retest

joelddiaz
joelddiaz reviewed Jul 8, 2020
View reviewed changes
Copy link
Contributor

@joelddiaz joelddiaz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Only high-level question I have is this: why are we not setting the Reason field on the conditions, and is there some unusual consequence for this decision?

@sjenning sjenning force-pushed the add-oidc-controller-status-handler branch from 541f4eb to 4ea8b15 Compare July 8, 2020 17:23
@sjenning
Copy link
Contributor Author

sjenning commented Jul 8, 2020

@joelddiaz added reasons

joelddiaz
joelddiaz reviewed Jul 9, 2020
View reviewed changes
Copy link
Contributor

@joelddiaz joelddiaz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ci-robot openshift-ci-robot added lgtm Indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Jul 9, 2020
@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

4 similar comments
@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@sjenning
Copy link
Contributor Author

sjenning commented Jul 9, 2020

gah, i can't copy and paste! pushing update...

@sjenning sjenning force-pushed the add-oidc-controller-status-handler branch from 4ea8b15 to 8d54516 Compare July 9, 2020 15:47
@openshift-ci-robot openshift-ci-robot removed the lgtm Indicates that a PR is ready to be merged. label Jul 9, 2020
joelddiaz
joelddiaz reviewed Jul 9, 2020
View reviewed changes
Copy link
Contributor

@joelddiaz joelddiaz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Jul 9, 2020
@openshift-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: joelddiaz, sjenning

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-robot openshift-merge-robot merged commit fb8f3f8 into openshift:master Jul 9, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@joelddiaz joelddiaz joelddiaz left review comments

Assignees

@dgoodwin dgoodwin

@joelddiaz joelddiaz

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
6 participants
@sjenning @openshift-bot @openshift-ci-robot @joelddiaz @dgoodwin @openshift-merge-robot