Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug 2005357: [4.8z] Fixes misuse of pod annotations during update event #753

Merged
merged 1 commit into from Sep 22, 2021

Conversation

trozet
Copy link
Contributor

@trozet trozet commented Sep 17, 2021

In the update pod logic, we pass the current pod event to
addLogicalPort. In addLogicalPort we assume that if the annotations
exist for the pod mac/ifaddr, then we use those and do not update
annotations on the pod. This assumption is invalid, because this event
may not be the current state of the pod. In other words we could have a
situation where:

  1. A pod add event comes we annotate with 10.0.0.2, assume OVN execute
    failure
  2. Before the annotate is done, the pod is modified in some other way
    signaling another pod update event
  3. A pod update event comes for 2, the pod is annotated with 10.0.0.3
    because this was an update to the original pod, before it was
    annotated with 10.0.0.2, assume OVN execute failure
  4. A pod update event comes for 1, since annotations existed, nothing is
    annotated and 10.0.0.2 is found to be used. OVN logical port is
    configured with 10.0.0.2. addLogicalPort succeeds.
  5. Now the pod has 10.0.0.3 annotated, and 10.0.0.2 in OVN. CNI openflow
    check will fail and the pod will never come up.

Signed-off-by: Tim Rozet trozet@redhat.com

- What this PR does and why is it needed

- Special notes for reviewers

- How to verify it

- Description for the changelog

In the update pod logic, we pass the current pod event to
addLogicalPort. In addLogicalPort we assume that if the annotations
exist for the pod mac/ifaddr, then we use those and do not update
annotations on the pod. This assumption is invalid, because this event
may not be the current state of the pod. In other words we could have a
situation where:

1. A pod add event comes we annotate with 10.0.0.2, assume OVN execute
   failure
2. Before the annotate is done, the pod is modified in some other way
   signaling another pod update event
3. A pod update event comes for 2, the pod is annotated with 10.0.0.3
   because this was an update to the original pod, before it was
   annotated with 10.0.0.2, assume OVN execute failure
4. A pod update event comes for 1, since annotations existed, nothing is
   annotated and 10.0.0.2 is found to be used. OVN logical port is
   configured with 10.0.0.2. addLogicalPort succeeds.
5. Now the pod has 10.0.0.3 annotated, and 10.0.0.2 in OVN. CNI openflow
   check will fail and the pod will never come up.

Signed-off-by: Tim Rozet <trozet@redhat.com>
@openshift-ci openshift-ci bot added the bugzilla/severity-medium Referenced Bugzilla bug's severity is medium for the branch this PR is targeting. label Sep 17, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 17, 2021

@trozet: This pull request references Bugzilla bug 2005357, which is invalid:

  • expected dependent Bugzilla bug 2004340 to be in one of the following states: VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), CLOSED (CURRENTRELEASE), but it is ON_QA instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

Bug 2005357: [4.8z] Fixes misuse of pod annotations during update event

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci bot added the bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. label Sep 17, 2021
@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 17, 2021
@abhat
Copy link
Contributor

abhat commented Sep 17, 2021

/bugzilla refresh

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 17, 2021

@abhat: This pull request references Bugzilla bug 2005357, which is invalid:

  • expected dependent Bugzilla bug 2004340 to be in one of the following states: VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), CLOSED (CURRENTRELEASE), but it is ON_QA instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@tssurya
Copy link
Contributor

tssurya commented Sep 17, 2021

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Sep 17, 2021
@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

5 similar comments
@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/bugzilla refresh

Recalculating validity in case the underlying Bugzilla bug has changed.

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 18, 2021

@openshift-bot: This pull request references Bugzilla bug 2005357, which is invalid:

  • expected dependent Bugzilla bug 2004340 to be in one of the following states: VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), CLOSED (CURRENTRELEASE), but it is ON_QA instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

/bugzilla refresh

Recalculating validity in case the underlying Bugzilla bug has changed.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@yingwang-0320
Copy link

/lgtm and /label qe-approved

@openshift-bot
Copy link
Contributor

/bugzilla refresh

Recalculating validity in case the underlying Bugzilla bug has changed.

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 19, 2021

@openshift-bot: This pull request references Bugzilla bug 2005357, which is invalid:

  • expected dependent Bugzilla bug 2004340 to be in one of the following states: VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), CLOSED (CURRENTRELEASE), but it is ON_QA instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

/bugzilla refresh

Recalculating validity in case the underlying Bugzilla bug has changed.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-bot
Copy link
Contributor

/bugzilla refresh

Recalculating validity in case the underlying Bugzilla bug has changed.

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 20, 2021

@openshift-bot: This pull request references Bugzilla bug 2005357, which is invalid:

  • expected dependent Bugzilla bug 2004340 to be in one of the following states: VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), CLOSED (CURRENTRELEASE), but it is ON_QA instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

/bugzilla refresh

Recalculating validity in case the underlying Bugzilla bug has changed.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@trozet
Copy link
Contributor Author

trozet commented Sep 20, 2021

/bugzilla refresh

@openshift-ci openshift-ci bot added bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. and removed bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. labels Sep 20, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 20, 2021

@trozet: This pull request references Bugzilla bug 2005357, which is valid.

6 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.8.z) matches configured target release for branch (4.8.z)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)
  • dependent bug Bugzilla bug 2004340 is in the state VERIFIED, which is one of the valid states (VERIFIED, RELEASE_PENDING, CLOSED (ERRATA), CLOSED (CURRENTRELEASE))
  • dependent Bugzilla bug 2004340 targets the "4.9.0" release, which is one of the valid target releases: 4.9.0
  • bug has dependents

No GitHub users were found matching the public email listed for the QA contact in Bugzilla (anusaxen@redhat.com), skipping review request.

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@anuragthehatter
Copy link

/bugzilla cc-qa

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 20, 2021

@anuragthehatter: An error was encountered querying GitHub for users with public email (anusaxen@redhat.com) for bug 2005357 on the Bugzilla server at https://bugzilla.redhat.com. No known errors were detected, please see the full error message for details.

Full error message. non-200 OK status code: 403 Forbidden body: "{\n \"documentation_url\": \"https://docs.github.com/en/free-pro-team@latest/rest/overview/resources-in-the-rest-api#secondary-rate-limits\",\n \"message\": \"You have exceeded a secondary rate limit. Please wait a few minutes before you try again.\"\n}\n"

Please contact an administrator to resolve this issue, then request a bug refresh with /bugzilla refresh.

In response to this:

/bugzilla cc-qa

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@anuragthehatter
Copy link

/lgtm
/label qe-approved

@openshift-ci openshift-ci bot added the qe-approved Signifies that QE has signed off on this PR label Sep 20, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 20, 2021

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: anuragthehatter, trozet, tssurya, yingwang-0320

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@hardys
Copy link

hardys commented Sep 20, 2021

[patch-manager] Approving for cherry-pick with a score of 1.70

@hardys hardys added the cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. label Sep 20, 2021
@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

12 similar comments
@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@trozet
Copy link
Contributor Author

trozet commented Sep 22, 2021

/retest-required

@openshift-bot
Copy link
Contributor

/retest-required

Please review the full test history for this PR and help us cut down flakes.

@openshift-merge-robot openshift-merge-robot merged commit da36f39 into openshift:release-4.8 Sep 22, 2021
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 22, 2021

@trozet: All pull requests linked via external trackers have merged:

Bugzilla bug 2005357 has been moved to the MODIFIED state.

In response to this:

Bug 2005357: [4.8z] Fixes misuse of pod annotations during update event

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. bugzilla/severity-medium Referenced Bugzilla bug's severity is medium for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. lgtm Indicates that a PR is ready to be merged. qe-approved Signifies that QE has signed off on this PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants