Skip to content

Add TX checksum offload for Windows XDP #5080

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 31 commits into from
May 19, 2025
Merged

Conversation

mtfriesen
Copy link
Contributor

@mtfriesen mtfriesen commented May 6, 2025

Description

Describe the purpose of and changes within this Pull Request.

This PR adds support for TX checksum offload for Windows XDP. The offload may not be supported for at least the following reasons:

  1. The version of XDP does not support the offload yet
  2. The interface the socket is bound to does not support the offload
  3. The offload is defined as experimental, so the version of XDP may no longer support the offload

Note the version of the XDP runtime in MsQuic is not being updated to a prerelease build with checksum support, so this code will not be practically enabled yet in CI/netperf/etc.

Testing

Do any existing tests cover this change? Are new tests needed?

We rely on the XDP interop tests for correctness, though this is not robust if the VMs running the interop test do not advertise checksum support. I've also manually run interop tests between XDP client and the full stack IOCP server in netperf, which we should eventually onboard both for performance and conformance coverage.

Documentation

Is there any documentation impact for this change?

Mostly self-explanatory implementation details.

Copy link

codecov bot commented May 6, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 86.94%. Comparing base (65d6f63) to head (b77a615).
Report is 5 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #5080      +/-   ##
==========================================
+ Coverage   86.47%   86.94%   +0.47%     
==========================================
  Files          59       59              
  Lines       18035    18035              
==========================================
+ Hits        15595    15680      +85     
+ Misses       2440     2355      -85     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ProjectsByJackHe
Copy link
Contributor

ProjectsByJackHe commented May 8, 2025

For clarity, the TX offloading capabilities were already there (set IP->headerCheckSum = 0) in datapath_raw_socket prior to this PR, correct?

Would a better title for this PR be: "Set correct socket options for TX checksum offloads"?

@mtfriesen
Copy link
Contributor Author

For clarity, the TX offloading capabilities were already there (set IP->headerCheckSum = 0) in datapath_raw_socket prior to this PR, correct?

Would a better title for this PR be: "Set correct socket options for TX checksum offloads"?

Until now, the common datapath_raw_socket conditionally computed the checksums based on the datapath capabilities, and the Windows XDP datapath never declared that capability. So this PR adds TX checksum offloads for Windows XDP.

@mtfriesen mtfriesen marked this pull request as ready for review May 9, 2025 19:55
@mtfriesen mtfriesen requested a review from a team as a code owner May 9, 2025 19:55
@csujedihy
Copy link
Contributor

csujedihy commented May 9, 2025

It looks like there are 2 sources of truth now for offload enablement.

  1. Interface->OffloadStatus.Transmit.TransportLayerXsum

  2. CxPlatDpRawIsL3TxXsumOffloadedOnQueue

1 is only used in QTIP code path. Is this expected?

@csujedihy
Copy link
Contributor

very nit: "Describe the purpose of and changes within this Pull Request." we probably don't need the placeholder in the PR describption.

@mtfriesen
Copy link
Contributor Author

It looks like there are 2 sources of truth now for offload enablement.

  1. Interface->OffloadStatus.Transmit.TransportLayerXsum
  2. CxPlatDpRawIsL3TxXsumOffloadedOnQueue

1 is only used in QTIP code path. Is this expected?

Thanks, I meant to update QTIP too, but got forgot in the chaos. Great catch.

@mtfriesen mtfriesen marked this pull request as draft May 14, 2025 13:35
@mtfriesen
Copy link
Contributor Author

Converting back to draft status while experimentation continues.

@mtfriesen mtfriesen marked this pull request as ready for review May 16, 2025 12:08
@mtfriesen mtfriesen enabled auto-merge (squash) May 16, 2025 12:08
Comment on lines +400 to +402
OptionLength < XDP_SIZEOF_CHECKSUM_CONFIGURATION_REVISION_1 ||
ChecksumConfig.Header.Revision != XDP_CHECKSUM_CONFIGURATION_REVISION_1 ||
ChecksumConfig.Header.Size < XDP_SIZEOF_CHECKSUM_CONFIGURATION_REVISION_1) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, if XDP starts returning a version 2, it breaks our support for the offload? If so, that doesn't seem like a great design.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Version 2 is by definition a breaking change, and version 2 has not yet been defined. The only legal action the application can take here is to ignore the payload.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that if we make it version 2, it's breaking, but should we instead of a different socket option to query version 2?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have a strong opinion, other than that XDP already consistently uses this pattern everywhere, and I don't see much justification to deviate here.

"XskSetSockopt(XSK_SOCKOPT_TX_OFFLOAD_CHECKSUM)");
}
} else {
QuicTraceEvent(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a duplicate log in the QUIC_FAILED case above.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure - GetTxOffloadConfig can return QUIC_SUCCEEDED and TxChecksumOffload can be false.

@csujedihy
Copy link
Contributor

There's a code defect (probably introduced by myself) that's not part of your PR. If we don't rely on these flags for xdp anymore, maybe delete these lines?

In datapath_raw_xdp_win.c:

    Interface->OffloadStatus.Transmit.NetworkLayerXsum = Xdp->SkipXsum;
    Interface->OffloadStatus.Transmit.NetworkLayerXsum = Xdp->SkipXsum;

The second line should be setting the transport layer xsum.

@mtfriesen mtfriesen merged commit ae93aad into main May 19, 2025
289 of 290 checks passed
@mtfriesen mtfriesen deleted the mtfriesen/win_xdp_tx_csum branch May 19, 2025 17:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants