New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[release-4.9] Bug 2054299: Lock the reassignment procedure during node deletion to avoid races #955
[release-4.9] Bug 2054299: Lock the reassignment procedure during node deletion to avoid races #955
Conversation
@kyrtapz: This pull request references Bugzilla bug 2053611, which is invalid:
Comment In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
@kyrtapz: This pull request references Bugzilla bug 2053611, which is invalid:
Comment In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
/retitle [WIP] Bug 2054299: Lock the reassignment procedure during node deletion to avoid races |
@kyrtapz: This pull request references Bugzilla bug 2054299, which is invalid:
Comment In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
/lgtm |
/bugzilla refresh |
@flavio-fernandes: This pull request references Bugzilla bug 2054299, which is valid. The bug has been moved to the POST state. 6 validation(s) were run on this bug
Requesting review from QA contact: In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
@kyrtapz: This pull request references Bugzilla bug 2054299, which is valid. 6 validation(s) were run on this bug
Requesting review from QA contact: In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
1 similar comment
@kyrtapz: This pull request references Bugzilla bug 2054299, which is valid. 6 validation(s) were run on this bug
Requesting review from QA contact: In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
/retitle Bug 2054299: Lock the reassignment procedure during node deletion to avoid races |
/assign @trozet |
/unhold |
/remove-work-in-progress |
/shrug |
/shrug cancel |
/retest-required Please review the full test history for this PR and help us cut down flakes. |
12 similar comments
/retest-required Please review the full test history for this PR and help us cut down flakes. |
/retest-required Please review the full test history for this PR and help us cut down flakes. |
/retest-required Please review the full test history for this PR and help us cut down flakes. |
/retest-required Please review the full test history for this PR and help us cut down flakes. |
/retest-required Please review the full test history for this PR and help us cut down flakes. |
/retest-required Please review the full test history for this PR and help us cut down flakes. |
/retest-required Please review the full test history for this PR and help us cut down flakes. |
/retest-required Please review the full test history for this PR and help us cut down flakes. |
/retest-required Please review the full test history for this PR and help us cut down flakes. |
/retest-required Please review the full test history for this PR and help us cut down flakes. |
/retest-required Please review the full test history for this PR and help us cut down flakes. |
/retest-required Please review the full test history for this PR and help us cut down flakes. |
/label qe-approved |
/label cherry-pick-approved |
/retest-required Please review the full test history for this PR and help us cut down flakes. |
6 similar comments
/retest-required Please review the full test history for this PR and help us cut down flakes. |
/retest-required Please review the full test history for this PR and help us cut down flakes. |
/retest-required Please review the full test history for this PR and help us cut down flakes. |
/retest-required Please review the full test history for this PR and help us cut down flakes. |
/retest-required Please review the full test history for this PR and help us cut down flakes. |
/retest-required Please review the full test history for this PR and help us cut down flakes. |
@kyrtapz: The following tests failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
@kyrtapz: All pull requests linked via external trackers have merged: Bugzilla bug 2054299 has been moved to the MODIFIED state. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
/cherry-pick release-4.8 |
@kyrtapz: new pull request created: #970 In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
deleteEgressNode
takes care of reassigning egress IPs that are hosted on that node when it gets deleted, becomes unreachable, the labelk8s.ovn.org/egress-assignable
gets removed.It is possible that when
deleteEgressNode
takes a long time to reassign the egress ips another one is called(e.g node becomes unreachable and thenNotReady
).Best case scenario the egressIP is reassigned twice, worst case the status doesn't get updated properly or the egress IP is not correctly assigned.
I do not think that this change brings any deadlock potential as this approach is already used in
addEgressNode
, nevertheless it would be good to focus on that while reviewing the change.I was able to reproduce the bug and verify the fix by creating 400 pods that match the egress IP assigned to
node
and then run:Signed-off-by: Patryk Diak pdiak@redhat.com