Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

operator: release leader lease lock on operator exit #14554

Merged
merged 1 commit into from
Jan 14, 2021
Merged

operator: release leader lease lock on operator exit #14554

merged 1 commit into from
Jan 14, 2021

Conversation

fristonio
Copy link
Member

Fixes #14546

The operator on receiving the termination signal will wait for the lease to be released and then exit.

Jan 07 18:42:21 k8s1 cilium-operator[4918]: level=debug msg="Controller func execution time: 1.809µs" name=update-cilium-nodes-pod-cidr subsys=controller uuid=cc0277ad-5270-4335-b8ce-eaacc95e1717
Jan 07 18:42:23 k8s1 systemd[1]: Stopping cilium operator...
Jan 07 18:42:23 k8s1 cilium-operator[4918]: level=info msg="Received termination signal. Shutting down" subsys=cilium-operator
Jan 07 18:42:23 k8s1 cilium-operator[4918]: level=info msg="Leader election lost" operator-id=k8s1-jUBZGmHGKZ subsys=cilium-operator
Jan 07 18:42:23 k8s1 systemd[1]: Stopped cilium operator.

@fristonio fristonio added release-note/bug This PR fixes an issue in a previous release of Cilium. area/operator Impacts the cilium-operator component labels Jan 7, 2021
@fristonio fristonio requested a review from a team as a code owner January 7, 2021 18:55
@maintainer-s-little-helper maintainer-s-little-helper bot added this to In progress in 1.10.0 Jan 7, 2021
christarazi
christarazi approved these changes Jan 7, 2021
View reviewed changes
Copy link
Member

@christarazi christarazi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

operator/api/server.go Show resolved Hide resolved
@fristonio fristonio force-pushed the pr/fristonio/operator-ha-exit-fix branch from 3ab77a1 to 8660a01 Compare January 8, 2021 08:41
aanm
aanm approved these changes Jan 8, 2021
View reviewed changes
Copy link
Member

@aanm aanm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small non-blocking nit.

operator/api/server.go Show resolved Hide resolved
@aanm aanm removed their assignment Jan 8, 2021
@maintainer-s-little-helper maintainer-s-little-helper bot added this to Needs backport from master in 1.9.2 Jan 8, 2021
@fristonio
operator: release leader lease lock on operator exit
afeac30 
* This commit fixes an issue in cilium-operator where the operator does
  not release the leader lease lock once it exits. This is caused due to
  the fact that cilium-operator exits preemptively without waiting for
  the operator to release the lock.

Signed-off-by: Deepesh Pathak <deepshpathak@gmail.com>
@fristonio fristonio force-pushed the pr/fristonio/operator-ha-exit-fix branch from 8660a01 to afeac30 Compare January 8, 2021 09:01
@fristonio
Copy link
Member Author

test-me-please

@fristonio
Copy link
Member Author

retest-gke

@christarazi christarazi removed their assignment Jan 8, 2021
@aanm aanm merged commit 7b72e77 into master Jan 14, 2021
@aanm aanm deleted the pr/fristonio/operator-ha-exit-fix branch January 14, 2021 08:07
@errordeveloper errordeveloper mentioned this pull request Jan 14, 2021
Merged
@joestringer joestringer moved this from Needs backport from master to Backport pending to v1.9 in 1.9.2 Jan 18, 2021
@joestringer
Copy link
Member

Looks like this fell through the cracks in 1.9 backports.

@errordeveloper reported in the linked PR:

@fristonio as you suggested, I will drop #14554 as it depends on a larger PR #14484.

Opting to drop the 1.9 backport label. @fristonio , if you think this is important to backport to v1.9 then please either initiate the backport or reach out in #launchpad on slack to coordinate.

@maintainer-s-little-helper maintainer-s-little-helper bot removed this from Backport pending to v1.9 in 1.9.2 Jan 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@christarazi christarazi christarazi approved these changes

@aanm aanm aanm approved these changes

Assignees
No one assigned
Labels
area/operator Impacts the cilium-operator component release-note/bug This PR fixes an issue in a previous release of Cilium.
Projects
No open projects
1.10.0
Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

CIlium operator HA lease lock is not released properly
5 participants
@fristonio @joestringer @christarazi @aanm @errordeveloper