Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Smaller MAX_PACKET_SIZE and better batch cache locality #816

Merged
merged 1 commit into from
Mar 10, 2024

Conversation

droe
Copy link
Contributor

@droe droe commented Mar 9, 2024

This is a very arguable change, feel free to reject. I'm seeing a consistent ~ 0.7 % send rate improvement with lower-end hardware at 10 GbE close to but not quite at max send rate.

  • Change how batch is laid out in memory: Since all access is one packet at a time and never all IPs or all lengths, lay out all fields pertaining to the same packet adjacent in memory, for sequential access (single array of packets with metadata instead of separate arrays for packet data, lengths and ips)
  • Reduce MAX_PACKET_SIZE from 4k to 2k minus metadata, which is still way above the standard MTU of 1500 that we probably don't want to exceed in order to avoid IP fragmentation

Extra benefits:

  • Arguably cleaner code for accessing the packets in batches (no manual offset calculations)
  • Single definition of MAX_PACKET_SIZE instead of the previous two definitions of MAX_PACKET_SIZE

Tested on the usual macOS Sonoma, FreeBSD 14 and Ubuntu 23.10 with and without netmap.

@zakird
Copy link
Member

zakird commented Mar 10, 2024

Thanks. Performance increase is slim, but it's a cleaner design for sure, so happy to merge.

@zakird zakird merged commit 7dce05c into zmap:main Mar 10, 2024
11 checks passed
@droe droe deleted the droe/batch-perf branch March 11, 2024 08:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants