Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[release-4.5] Bug 1810574: CNI: Don't wait for missing pods on DEL #277

Merged
merged 1 commit into from Jun 11, 2020
Merged

[release-4.5] Bug 1810574: CNI: Don't wait for missing pods on DEL #277

merged 1 commit into from Jun 11, 2020

Conversation

openshift-cherrypick-robot
Copy link

This is an automated cherry-pick of #274

/assign luis5tb

@dulek
CNI: Don't wait for missing pods on DEL
5ae8f69 
We've seen some issues related to the fact that on CNI DEL we wait for
pod annotation to appear in CNI registry:

1. Overloaded kuryr-daemon HTTP server, because many stale DEL requests
   from CRI will saturate the number of free connections, causing CNI to
   answer requests extremely slowly.
2. Some CRI's will not delete network namespace before getting a
   successful CNI DEL. If we'll assign the same IP (or vlan) to a next
   pod, we might end up with an IP or vlan conflict and NetlinkError.

This commit makes sure we only wait for VIFs up to 5 seconds on DEL
requests, enough time for watcher to populate registry on restart. Also
some code got moved around to make implementating the above simpler and
some logs got clarified to make debugging of such issues easier later
on.

Change-Id: I9221b6bc9166597837a4b53382862aa6c6f3e94c
Closes-Bug: 1882083
Related-Bug: 1854928
@openshift-cherrypick-robot openshift-cherrypick-robot mentioned this pull request Jun 11, 2020
Merged
@openshift-ci-robot
Copy link

@openshift-cherrypick-robot: Bugzilla bug 1846225 has been cloned as Bugzilla bug 1846262. Retitling PR to link against new bug.
/retitle [release-4.5] Bug 1846262: CNI: Don't wait for missing pods on DEL

In response to this:

[release-4.5] Bug 1846225: CNI: Don't wait for missing pods on DEL

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci-robot openshift-ci-robot changed the title [release-4.5] Bug 1846225: CNI: Don't wait for missing pods on DEL [release-4.5] Bug 1846262: CNI: Don't wait for missing pods on DEL Jun 11, 2020
@openshift-ci-robot openshift-ci-robot added bugzilla/severity-medium Referenced Bugzilla bug's severity is medium for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. labels Jun 11, 2020
@openshift-ci-robot
Copy link

@openshift-cherrypick-robot: This pull request references Bugzilla bug 1846262, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

6 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.5.0) matches configured target release for branch (4.5.0)
  • bug is in the state NEW, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)
  • dependent bug Bugzilla bug 1846225 is in the state MODIFIED, which is one of the valid states (MODIFIED, ON_QA, VERIFIED)
  • dependent Bugzilla bug 1846225 targets the "4.6.0" release, which is one of the valid target releases: 4.6.0, 4.6.z
  • bug has dependents

In response to this:

[release-4.5] Bug 1846262: CNI: Don't wait for missing pods on DEL

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@luis5tb
Copy link
Contributor

luis5tb commented Jun 11, 2020

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Jun 11, 2020
@openshift-ci-robot
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: luis5tb, openshift-cherrypick-robot

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 11, 2020
@luis5tb luis5tb changed the title [release-4.5] Bug 1846262: CNI: Don't wait for missing pods on DEL [release-4.5] Bug 1810574: CNI: Don't wait for missing pods on DEL Jun 11, 2020
@openshift-ci-robot openshift-ci-robot removed the bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. label Jun 11, 2020
@openshift-ci-robot
Copy link

@openshift-cherrypick-robot: This pull request references Bugzilla bug 1810574, which is invalid:

  • expected dependent Bugzilla bug 1819129 to be in one of the following states: MODIFIED, ON_QA, VERIFIED, but it is CLOSED (NOTABUG) instead
  • expected dependent Bugzilla bug 1819129 to target a release in 4.6.0, 4.6.z, but it targets "4.5.0" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

[release-4.5] Bug 1810574: CNI: Don't wait for missing pods on DEL

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci-robot openshift-ci-robot added the bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. label Jun 11, 2020
@luis5tb
Copy link
Contributor

luis5tb commented Jun 11, 2020

/bugzilla refresh

@openshift-ci-robot
Copy link

@luis5tb: This pull request references Bugzilla bug 1810574, which is invalid:

  • expected dependent Bugzilla bug 1819129 to be in one of the following states: MODIFIED, ON_QA, VERIFIED, but it is CLOSED (NOTABUG) instead
  • expected dependent Bugzilla bug 1819129 to target a release in 4.6.0, 4.6.z, but it targets "4.5.0" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@luis5tb
Copy link
Contributor

luis5tb commented Jun 11, 2020

/bugzilla refresh

@openshift-ci-robot
Copy link

@luis5tb: This pull request references Bugzilla bug 1810574, which is invalid:

  • expected Bugzilla bug 1810574 to depend on a bug targeting a release in 4.6.0, 4.6.z and in one of the following states: MODIFIED, ON_QA, VERIFIED, but no dependents were found

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@luis5tb
Copy link
Contributor

luis5tb commented Jun 11, 2020

/bugzilla refresh

@openshift-ci-robot
Copy link

@luis5tb: This pull request references Bugzilla bug 1810574, which is invalid:

  • expected Bugzilla bug 1810574 to depend on a bug targeting a release in 4.6.0, 4.6.z and in one of the following states: MODIFIED, ON_QA, VERIFIED, but no dependents were found

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@luis5tb
Copy link
Contributor

luis5tb commented Jun 11, 2020

/bugzilla refresh

@openshift-ci-robot
Copy link

@luis5tb: This pull request references Bugzilla bug 1810574, which is valid. The bug has been moved to the POST state.

6 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.5.0) matches configured target release for branch (4.5.0)
  • bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)
  • dependent bug Bugzilla bug 1846225 is in the state ON_QA, which is one of the valid states (MODIFIED, ON_QA, VERIFIED)
  • dependent Bugzilla bug 1846225 targets the "4.6.0" release, which is one of the valid target releases: 4.6.0, 4.6.z
  • bug has dependents

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci-robot openshift-ci-robot added bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. and removed bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. labels Jun 11, 2020
@knobunc knobunc added the staff-eng-approved Indicates a release branch PR has been approved by a staff engineer (formerly group/pillar lead). label Jun 11, 2020
@openshift-merge-robot openshift-merge-robot merged commit 16ea79a into openshift:release-4.5 Jun 11, 2020
@openshift-ci-robot
Copy link

@openshift-cherrypick-robot: All pull requests linked via external trackers have merged: openshift/kuryr-kubernetes#277, openshift/kuryr-kubernetes#244, openshift/kuryr-kubernetes#179. Bugzilla bug 1810574 has been moved to the MODIFIED state.

In response to this:

[release-4.5] Bug 1810574: CNI: Don't wait for missing pods on DEL

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@luis5tb
Copy link
Contributor

luis5tb commented Jun 11, 2020

/cherry-pick release-4.4

@openshift-cherrypick-robot
Copy link
Author

@luis5tb: #277 failed to apply on top of branch "release-4.4":

error: Failed to merge in the changes.
Using index info to reconstruct a base tree...
M	kuryr_kubernetes/cni/binding/nested.py
M	kuryr_kubernetes/cni/daemon/service.py
M	kuryr_kubernetes/cni/plugins/k8s_cni_registry.py
Falling back to patching base and 3-way merge...
Auto-merging kuryr_kubernetes/cni/plugins/k8s_cni_registry.py
CONFLICT (content): Merge conflict in kuryr_kubernetes/cni/plugins/k8s_cni_registry.py
Auto-merging kuryr_kubernetes/cni/daemon/service.py
Auto-merging kuryr_kubernetes/cni/binding/nested.py
Patch failed at 0001 CNI: Don't wait for missing pods on DEL

In response to this:

/cherry-pick release-4.4

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

MaysaMacedo pushed a commit to MaysaMacedo/kuryr-kubernetes-1 that referenced this pull request Jul 1, 2021
@openshift-merge-robot
Merge pull request openshift#277 from danehans/proxy_reconciler_readi…
566e1ba 
…ness

Adds readinessEndpoint validation for Proxy API type.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dulek dulek Awaiting requested review from dulek

@MaysaMacedo MaysaMacedo Awaiting requested review from MaysaMacedo

Assignees

@luis5tb luis5tb

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. bugzilla/severity-medium Referenced Bugzilla bug's severity is medium for the branch this PR is targeting. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. lgtm Indicates that a PR is ready to be merged. staff-eng-approved Indicates a release branch PR has been approved by a staff engineer (formerly group/pillar lead).
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
6 participants
@openshift-cherrypick-robot @openshift-ci-robot @luis5tb @knobunc @openshift-merge-robot @dulek