Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

service: Update for L7 LB while locked #31744

Merged
merged 1 commit into from Apr 9, 2024

Conversation

jrajahalme
Copy link
Member

Keep service manager locked while updating services after L7 LB redirection updates. This removes the race where a service would be re-upserted while having been (concurrently) removed.

Fixes: #18894

Reported-by: Jussi Maki joamaki@isovalent.com

Fixed a race condition in service updates for L7 LB.

@jrajahalme jrajahalme added kind/bug This is a bug in the Cilium logic. release-note/bug This PR fixes an issue in a previous release of Cilium. needs-backport/1.13 This PR / issue needs backporting to the v1.13 branch needs-backport/1.14 This PR / issue needs backporting to the v1.14 branch needs-backport/1.15 This PR / issue needs backporting to the v1.15 branch labels Apr 3, 2024
@jrajahalme jrajahalme requested review from a team as code owners April 3, 2024 11:53
@maintainer-s-little-helper maintainer-s-little-helper bot added this to Needs backport from main in 1.15.4 Apr 3, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot added this to Needs backport from main in 1.14.10 Apr 3, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot added this to Needs backport from main in 1.13.15 Apr 3, 2024
@jrajahalme jrajahalme marked this pull request as draft April 3, 2024 11:54
@jrajahalme jrajahalme added the backport/author The backport will be carried out by the author of the PR. label Apr 3, 2024
@jrajahalme
Copy link
Member Author

Labeled for author backport due to recent churn in the affected code.

@jrajahalme
Copy link
Member Author

/test

@jrajahalme jrajahalme marked this pull request as ready for review April 3, 2024 15:12
Keep service manager locked while reupserting services after L7 LB
redirection updates. This removes the race where a service would be
re-upserted while having been (concurrently) removed.

Fixes: cilium#18894

Reported-by: Jussi Maki <joamaki@isovalent.com>
Signed-off-by: Jarno Rajahalme <jarno@isovalent.com>
@jrajahalme
Copy link
Member Author

/test

Copy link
Member

@mhofstetter mhofstetter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice fix and cleanup - thanks Jarno!

@mhofstetter mhofstetter removed the request for review from derailed April 5, 2024 07:43
@ysksuzuki
Copy link
Member

This PR is linked to #18894, but I can't find the report for the problem this PR fixes. Could you give me a pointer to the report?

@jrajahalme
Copy link
Member Author

This PR is linked to #18894, but I can't find the report for the problem this PR fixes. Could you give me a pointer to the report?

#18894 introduced the racy code that is fixed here. Original code gets copy of a service and then adds it again to account for the changes in L7 LB service registrations. The problem is that this was done without holding a lock, making it possible that service that has been removed after the copy was created is erroneously added back.

@ysksuzuki
Copy link
Member

I see. Thanks for the explanation!

Copy link
Member

@ysksuzuki ysksuzuki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@maintainer-s-little-helper maintainer-s-little-helper bot added the ready-to-merge This PR has passed all tests and received consensus from code owners to merge. label Apr 9, 2024
@jrajahalme jrajahalme added this pull request to the merge queue Apr 9, 2024
Merged via the queue into cilium:main with commit d913b62 Apr 9, 2024
62 checks passed
@jrajahalme jrajahalme deleted the l7-lb-service-lock branch April 9, 2024 12:01
@jrajahalme jrajahalme added backport-pending/1.15 The backport for Cilium 1.15.x for this PR is in progress. and removed needs-backport/1.15 This PR / issue needs backporting to the v1.15 branch labels Apr 9, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Needs backport from main to Backport pending to v1.15 in 1.15.4 Apr 9, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Needs backport from main to Backport pending to v1.15 in 1.15.4 Apr 9, 2024
@jrajahalme jrajahalme added backport-pending/1.14 The backport for Cilium 1.14.x for this PR is in progress. and removed needs-backport/1.14 This PR / issue needs backporting to the v1.14 branch labels Apr 9, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Needs backport from main to Backport pending to v1.14 in 1.14.10 Apr 9, 2024
@jrajahalme jrajahalme added backport-pending/1.13 The backport for Cilium 1.13.x for this PR is in progress. and removed needs-backport/1.13 This PR / issue needs backporting to the v1.13 branch labels Apr 9, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Needs backport from main to Backport pending to v1.13 in 1.13.15 Apr 9, 2024
@github-actions github-actions bot added backport-done/1.14 The backport for Cilium 1.14.x for this PR is done. and removed backport-pending/1.14 The backport for Cilium 1.14.x for this PR is in progress. labels Apr 11, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Backport pending to v1.14 to Backport done to v1.14 in 1.14.10 Apr 11, 2024
@github-actions github-actions bot added backport-done/1.13 The backport for Cilium 1.13.x for this PR is done. and removed backport-pending/1.13 The backport for Cilium 1.13.x for this PR is in progress. labels Apr 11, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot removed this from Backport pending to v1.13 in 1.13.15 Apr 11, 2024
@github-actions github-actions bot added backport-done/1.15 The backport for Cilium 1.15.x for this PR is done. and removed backport-pending/1.15 The backport for Cilium 1.15.x for this PR is in progress. labels Apr 11, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot removed this from Backport pending to v1.15 in 1.15.4 Apr 11, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot added this to Backport done to v1.13 in 1.13.15 Apr 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport/author The backport will be carried out by the author of the PR. backport-done/1.13 The backport for Cilium 1.13.x for this PR is done. backport-done/1.14 The backport for Cilium 1.14.x for this PR is done. backport-done/1.15 The backport for Cilium 1.15.x for this PR is done. kind/bug This is a bug in the Cilium logic. ready-to-merge This PR has passed all tests and received consensus from code owners to merge. release-note/bug This PR fixes an issue in a previous release of Cilium.
Projects
No open projects
1.13.15
Backport done to v1.13
1.14.10
Backport done to v1.14
Development

Successfully merging this pull request may close these issues.

None yet

3 participants