Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug 1777144: packageserver update fails to adopt APIService #1171

Merged
merged 1 commit into from Jan 31, 2020

Conversation

exdx
Copy link
Member

@exdx exdx commented Dec 4, 2019

Description of the change:
This PR does 2 things to address the issue where updating the packageserver fails with a unable to adopt APIService message. One is add APIService to the cleanup owner references logic that OLM performs on startup to ensure owner references on APIService objects are correct. The other is adding verification on the labels of the APIService object to make sure that the label olm.owner:packageserver is set.

Motivation for the change:
OLM already checks several types for owner reference mismatches: mainly rbac types like roles and role bindings. Adding the check for APIService owner references will hopefully resolve the issue during cluster upgrade where the packageserver blocks upgrade because it is unable to set the ownership for its api service. The packageserver CSV fails to resolve with message: unable to adopt APIService and this causes cluster upgrade to fail. By checking APIService references on startup this could potentially resolve the issue. Owner references are also established by labels on OLM objects, so adding a check for olm.owner:packageserver should also help.

Reviewer Checklist

  • Implementation matches the proposed design, or proposal is updated to match implementation
  • Sufficient unit test coverage
  • Sufficient end-to-end test coverage
  • Docs updated or added to /docs
  • Commit messages sensible and descriptive

@exdx exdx requested a review from dinhxuanvu December 4, 2019 22:04
@openshift-ci-robot openshift-ci-robot added the bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. label Dec 4, 2019
@openshift-ci-robot
Copy link
Collaborator

@exdx: This pull request references Bugzilla bug 1777144, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

Bug 1777144: Add api services to OLM cleanup logic

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci-robot openshift-ci-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Dec 4, 2019
@openshift-ci-robot openshift-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Dec 5, 2019
@openshift-ci-robot openshift-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Dec 5, 2019
@awgreene
Copy link
Member

awgreene commented Dec 6, 2019

Nice work on this @exdx , could you add a rewrite the commit message to include:

  • A short description of the bug you are fixing.
  • How your solution addresses the issue.

@awgreene
Copy link
Member

awgreene commented Dec 6, 2019

@exdx could you also merge your commits into a single commit.

@awgreene
Copy link
Member

awgreene commented Dec 6, 2019

@exdx I saw that the PR description included:

By checking api service references on startup this could potentially resolve the issue.

Were you able to test this and prove that the proposed solution will fix the bug?

Copy link
Member

@awgreene awgreene left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, nice work!

cmd/olm/cleanup.go Outdated Show resolved Hide resolved
cmd/olm/cleanup.go Outdated Show resolved Hide resolved
@exdx
Copy link
Member Author

exdx commented Dec 7, 2019

Thanks for the review @awgreene! Will update to incorporate your great feedback.

@exdx exdx changed the title Bug 1777144: Add api services to OLM cleanup logic Bug 1777144: packageserver update fails to adopt APIService Dec 9, 2019
@openshift-ci-robot
Copy link
Collaborator

@exdx: This pull request references Bugzilla bug 1777144, which is valid.

In response to this:

Bug 1777144: packageserver update fails to adopt APIService

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

cmd/olm/cleanup.go Outdated Show resolved Hide resolved
@exdx exdx force-pushed the bug-1777144 branch 2 times, most recently from f7d5f5d to ea8a632 Compare December 9, 2019 20:15
cmd/olm/cleanup.go Outdated Show resolved Hide resolved
@exdx exdx force-pushed the bug-1777144 branch 2 times, most recently from ba409b9 to 77a5e9f Compare January 14, 2020 21:30
@exdx
Copy link
Member Author

exdx commented Jan 15, 2020

/retest

1 similar comment
@exdx
Copy link
Member Author

exdx commented Jan 15, 2020

/retest

Copy link
Member

@njhale njhale left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good, mostly some nits.

/hold cancel

cmd/olm/cleanup_test.go Outdated Show resolved Hide resolved
pkg/controller/operators/olm/apiservices.go Outdated Show resolved Hide resolved
pkg/controller/operators/olm/apiservices.go Outdated Show resolved Hide resolved
cmd/olm/cleanup.go Outdated Show resolved Hide resolved
pkg/controller/operators/olm/apiservices.go Outdated Show resolved Hide resolved
@openshift-ci-robot openshift-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 15, 2020
@exdx
Copy link
Member Author

exdx commented Jan 21, 2020

It looks like the packageserver owner labels to the api service should already be set when creating the APIService (we were not sure originally if they were being set correctly). That being said, the addition of the cleanup code should still be helpful in resolving the original bug on cluster upgrade?

Also the bug was targeting a 4.2.z release for a 4.1 -> 4.2 upgrade, need to check that this all still applies.

…e to adopt APIService status message. This is a result of ownership mismatches between the packageserver (owner) and the APIService (the owned object). This can be resolved by setting the actual k8s owner references or the label-based references that OLM uses. This commit ensures that ownership references for the APIService are cleaned up on OLM restart and that the olm.owner:packageserver label is set on the corresponding APIService object.
@exdx
Copy link
Member Author

exdx commented Jan 21, 2020

/retest

2 similar comments
@exdx
Copy link
Member Author

exdx commented Jan 21, 2020

/retest

@exdx
Copy link
Member Author

exdx commented Jan 21, 2020

/retest

@exdx exdx requested a review from njhale January 21, 2020 23:10
@openshift-ci-robot
Copy link
Collaborator

@exdx: This pull request references Bugzilla bug 1777144, which is valid.

In response to this:

Bug 1777144: packageserver update fails to adopt APIService

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@ecordell
Copy link
Member

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Jan 30, 2020
@openshift-ci-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ecordell, exdx, njhale

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

3 similar comments
@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-merge-robot openshift-merge-robot merged commit 224f57d into operator-framework:master Jan 31, 2020
@openshift-ci-robot
Copy link
Collaborator

@exdx: All pull requests linked via external trackers have merged. Bugzilla bug 1777144 has been moved to the MODIFIED state.

In response to this:

Bug 1777144: packageserver update fails to adopt APIService

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@exdx
Copy link
Member Author

exdx commented Jan 31, 2020

/cherry-pick release-4.3

@openshift-cherrypick-robot

@exdx: new pull request created: #1270

In response to this:

/cherry-pick release-4.3

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@exdx
Copy link
Member Author

exdx commented Jan 31, 2020

/cherry-pick release-4.2

@openshift-cherrypick-robot

@exdx: new pull request created: #1271

In response to this:

/cherry-pick release-4.2

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. lgtm Indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants