Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ipsec: Debug info for transient IPsec upgrade drops #32240

Merged

Conversation

pchaigno
Copy link
Member

@pchaigno pchaigno commented Apr 29, 2024

During IPsec upgrades, we may have to temporarily remove some XFRM states due to conflicts with the new states and because the Linux API doesn't enable us to perform this atomically as we do for XFRM policies.

This temporary removal should be very short but can still cause drops under heavy throughput. This pull request logs the duration of the removal and the number of drops that happened.

I validated the behavior on v1.13. The removal lasts 200-900µs for every remote node.

@pchaigno pchaigno added area/encryption Impacts encryption support such as IPSec, WireGuard, or kTLS. release-note/misc This PR makes changes that have no direct user impact. needs-backport/1.13 needs-backport/1.14 This PR / issue needs backporting to the v1.14 branch feature/ipsec Relates to Cilium's IPsec feature needs-backport/1.15 This PR / issue needs backporting to the v1.15 branch labels Apr 29, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot added this to Needs backport from main in 1.15.5 Apr 29, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot added this to Needs backport from main in 1.14.11 Apr 29, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot added this to Needs backport from main in 1.13.16 Apr 29, 2024
Context: During IPsec upgrades, we may have to temporarily remove some
XFRM states due to conflicts with the new states and because the Linux
API doesn't enable us to perform this atomically as we do for XFRM
policies.

This commit moves this removal logic to its own function. That logic
will grow in subsequent commits as I'll add debugging information to the
log message.

This commit doesn't make any functional changes.

Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com>
Context: During IPsec upgrades, we may have to temporarily remove some
XFRM states due to conflicts with the new states and because the Linux
API doesn't enable us to perform this atomically as we do for XFRM
policies.

This temporary removal should be very short but can still cause drops
under heavy throughput. This commit logs the duration of the removal so
we can validate that it's actually always short and estimate the impact
on packet drops.

Note the log message will now be displayed only once the XFRM state is
re-added, instead of when it's removed like before.

Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com>
Context: During IPsec upgrades, we may have to temporarily remove some
XFRM states due to conflicts with the new states and because the Linux
API doesn't enable us to perform this atomically as we do for XFRM
policies.

This temporary removal should be very short but can still cause drops
under heavy throughput. This commit logs how many such drops happened.

Signed-off-by: Paul Chaignon <paul.chaignon@gmail.com>
@pchaigno pchaigno force-pushed the debug-info-for-transient-ipsec-upgrade-drops branch from ba8a427 to 9846d73 Compare April 29, 2024 16:17
@pchaigno
Copy link
Member Author

/test

@pchaigno pchaigno marked this pull request as ready for review April 30, 2024 21:08
@pchaigno pchaigno requested a review from a team as a code owner April 30, 2024 21:08
@pchaigno pchaigno enabled auto-merge April 30, 2024 21:16
@pchaigno pchaigno added this pull request to the merge queue May 2, 2024
Merged via the queue into cilium:main with commit 76d6670 May 2, 2024
64 checks passed
@pchaigno pchaigno deleted the debug-info-for-transient-ipsec-upgrade-drops branch May 2, 2024 16:07
@pippolo84 pippolo84 mentioned this pull request May 6, 2024
14 tasks
@pippolo84 pippolo84 added backport-pending/1.15 The backport for Cilium 1.15.x for this PR is in progress. and removed needs-backport/1.15 This PR / issue needs backporting to the v1.15 branch labels May 6, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Needs backport from main to Backport pending to v1.15 in 1.15.5 May 6, 2024
@pippolo84 pippolo84 mentioned this pull request May 6, 2024
7 tasks
@pippolo84 pippolo84 added the backport-pending/1.14 The backport for Cilium 1.14.x for this PR is in progress. label May 6, 2024
@pippolo84 pippolo84 removed the needs-backport/1.14 This PR / issue needs backporting to the v1.14 branch label May 6, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Needs backport from main to Backport pending to v1.14 in 1.14.11 May 6, 2024
@pippolo84 pippolo84 mentioned this pull request May 6, 2024
6 tasks
@pippolo84 pippolo84 added backport-pending/1.13 The backport for Cilium 1.13.x for this PR is in progress. and removed needs-backport/1.13 labels May 6, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Needs backport from main to Backport pending to v1.13 in 1.13.16 May 6, 2024
@github-actions github-actions bot added backport-done/1.15 The backport for Cilium 1.15.x for this PR is done. backport-done/1.14 The backport for Cilium 1.14.x for this PR is done. and removed backport-pending/1.14 The backport for Cilium 1.14.x for this PR is in progress. backport-pending/1.15 The backport for Cilium 1.15.x for this PR is in progress. labels May 8, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Backport pending to v1.15 to Backport done to v1.15 in 1.15.5 May 8, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Backport pending to v1.15 to Backport done to v1.15 in 1.15.5 May 8, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot moved this from Backport pending to v1.14 to Backport done to v1.14 in 1.14.11 May 8, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot removed this from Backport done to v1.15 in 1.15.5 May 8, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot removed this from Backport done to v1.14 in 1.14.11 May 8, 2024
@github-actions github-actions bot added backport-done/1.13 The backport for Cilium 1.13.x for this PR is done. and removed backport-pending/1.13 The backport for Cilium 1.13.x for this PR is in progress. labels May 8, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot added this to Backport done to v1.15 in 1.15.5 May 8, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot added this to Backport done to v1.14 in 1.14.11 May 8, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot removed this from Backport pending to v1.13 in 1.13.16 May 8, 2024
@maintainer-s-little-helper maintainer-s-little-helper bot added this to Backport done to v1.13 in 1.13.16 May 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/encryption Impacts encryption support such as IPSec, WireGuard, or kTLS. backport-done/1.13 The backport for Cilium 1.13.x for this PR is done. backport-done/1.14 The backport for Cilium 1.14.x for this PR is done. backport-done/1.15 The backport for Cilium 1.15.x for this PR is done. feature/ipsec Relates to Cilium's IPsec feature release-note/misc This PR makes changes that have no direct user impact.
Projects
No open projects
1.13.16
Backport done to v1.13
1.14.11
Backport done to v1.14
1.15.5
Backport done to v1.15
Status: Released
Status: Released
Status: Released
Development

Successfully merging this pull request may close these issues.

None yet

3 participants